-- Logs begin at Mon 2022-05-23 14:39:16 UTC, end at Mon 2022-05-23 16:00:28 UTC. -- May 23 15:20:04.330706 dv-cinder-840678-12-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 23 15:20:04.340005 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 23 15:20:04.342044 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 23 15:20:04 2022] *** May 23 15:20:04.342044 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 23 15:20:04.342044 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 23 15:20:04.342044 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: nodename: dv-cinder-840678-12-iscsi May 23 15:20:04.342044 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: machine: x86_64 May 23 15:20:04.342044 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: clock source: unix May 23 15:20:04.342044 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: pcre jit disabled May 23 15:20:04.342044 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: detected number of CPU cores: 4 May 23 15:20:04.342044 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: current working directory: / May 23 15:20:04.342044 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: detected binary path: /usr/bin/uwsgi-core May 23 15:20:04.342742 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: your processes number limit is 47680 May 23 15:20:04.342742 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: your memory page size is 4096 bytes May 23 15:20:04.342742 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: detected max file descriptor number: 2048 May 23 15:20:04.342742 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: lock engine: pthread robust mutexes May 23 15:20:04.342742 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: thunder lock: enabled May 23 15:20:04.342742 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 23 15:20:04.342742 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 23 15:20:04.387540 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: Python main interpreter initialized at 0x55b7ae4a29c0 May 23 15:20:04.387540 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: python threads support enabled May 23 15:20:04.387540 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: your server socket listen backlog is limited to 100 connections May 23 15:20:04.387540 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: your mercy for graceful operations on workers is 90 seconds May 23 15:20:04.387897 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: mapped 403077 bytes (393 KB) for 2 cores May 23 15:20:04.387975 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: *** Operational MODE: preforking *** May 23 15:20:04.388233 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: *** uWSGI is running in multiple interpreter mode *** May 23 15:20:04.388233 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: spawned uWSGI master process (pid: 80585) May 23 15:20:04.388435 dv-cinder-840678-12-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 23 15:20:04.391489 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: spawned uWSGI worker 1 (pid: 80598, cores: 1) May 23 15:20:04.391489 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: spawned uWSGI worker 2 (pid: 80599, cores: 1) May 23 15:20:04.391489 dv-cinder-840678-12-iscsi devstack@keystone.service[80585]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 23 15:20:10.077580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:20:10.077580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: from cryptography.utils import int_from_bytes May 23 15:20:10.078366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:20:10.078366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: from cryptography.utils import int_from_bytes May 23 15:20:10.115532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:20:10.115532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: from cryptography.utils import int_from_bytes May 23 15:20:10.116061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:20:10.116061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: from cryptography.utils import int_from_bytes May 23 15:20:10.127703 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 15:20:10.127901 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 15:20:10.128099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] command line args: [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 15:20:10.128371 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 15:20:10.128552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ================================================================================ {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 15:20:10.128807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] admin_token = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.129053 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] config_dir = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.129266 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.129454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] config_source = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.129638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] debug = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.129837 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.130729 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.130729 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.130729 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.130938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.131062 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] list_limit = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.131249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] log_config_append = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.131431 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.131611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] log_dir = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.131791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] log_file = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.132026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.132275 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.132528 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.132776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.132955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.133225 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.133396 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.133643 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.133878 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.134124 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.134381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.134873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.135015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.135269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.135548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.136036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.136221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] publish_errors = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.136455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.136687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.136867 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.137148 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.137387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.137623 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] standard_threads = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.137853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.138097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.138275 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.138455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] use_journal = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.138632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] use_json = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.138809 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] use_stderr = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.138987 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] use_syslog = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.139217 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.139472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.139711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.139970 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.140209 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.140458 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.140696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.140943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.141212 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.141463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.141711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.141961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.142196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.142443 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.142685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.142932 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.143181 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.143418 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.143671 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.143925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.144169 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.144406 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.144659 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.144922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.145203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.145455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.145689 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.145920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.146152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.146387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.146634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.146883 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.147114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.147347 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.147594 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.147841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.148079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.148326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.148558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.148793 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.149027 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.149283 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.149515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.149746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.150775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.150775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.151034 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.151267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.151505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.151736 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.151973 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.152208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.152458 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.152689 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.152927 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.153191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.153431 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.153667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.153905 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.154136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.154389 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.154628 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.154867 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.155100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.155362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.155600 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.155833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.156073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.156312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.156583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.156822 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.157087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.157341 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.157581 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.157819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.158055 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.158305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.158545 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.158781 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.159024 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.159264 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.159502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.159739 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.159974 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.160210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.160472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.160714 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.160956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.161218 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.161463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.161701 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.161936 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.162173 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.162403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.162632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.162711 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 15:20:10.162853 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 15:20:10.162932 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.163046 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] command line args: [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 15:20:10.163126 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.163310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.163388 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 15:20:10.163489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ================================================================================ {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 15:20:10.163574 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.163742 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] admin_token = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.163819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.163985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] config_dir = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.164074 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.164175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.164260 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.164340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] config_source = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.164473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.164558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] debug = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.164721 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.164852 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.164951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.165093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.165199 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.165319 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] auth.external = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.165442 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.166515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.166615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.166728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.166864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.166953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.167073 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_config_append = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.167195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] auth.password = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.167273 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.167396 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] auth.token = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.167474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_dir = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.167474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_file = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.167662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.167739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.167856 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.167931 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.168003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.168130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.168207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.168286 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.168411 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.168492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.168622 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.168699 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.168831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.168956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.169034 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] credential.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.169172 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.169249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.169365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.169454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.169536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.169646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.169761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.169843 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.169961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.170054 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.170129 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.170202 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.170319 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.170394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.170466 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.170615 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.170853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.170933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.171052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] publish_errors = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.171135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.171265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.171375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.171491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.171638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.171732 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.171886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.171972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.172115 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.172198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.172347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] standard_threads = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.172502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.172584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.172656 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.172799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.172885 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.172959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.173077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.173160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] use_journal = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.173321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] use_json = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.173399 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.173519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] use_stderr = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.173594 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.173679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] use_syslog = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.173785 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.173871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:20:10.174026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.174107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.174283 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.174371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.174522 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.174608 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.174779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.174860 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.175019 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.175107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.175252 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.175343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.175489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.175567 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.175720 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.175804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.175963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.176050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.176198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.176277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.176435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.176514 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.176666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.176746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.176900 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.176978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.177154 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.177248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.177399 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.177480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.177644 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.177724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.177874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.177960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.178106 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.178177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.178347 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.178478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.178596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.178671 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.178807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.178887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.179058 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.179142 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.179311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.179391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.179812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.179898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.180058 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.180162 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.180286 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.180464 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.180552 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.180704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.180784 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.180938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.181019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.181256 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.181616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.181735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.181820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.181896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.181971 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.182045 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.182175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.182258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.182421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.182502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.182652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.182731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.182890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.182971 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.183137 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.183222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.183367 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.183458 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.183602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.183682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.183833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.183914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.184062 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.184142 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.184299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.184379 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.184527 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.184607 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.184755 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.184836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.184989 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.185125 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.185258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.185342 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.185488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.185574 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.185724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.185805 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.185950 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.186031 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.186184 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.186264 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.186421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.186508 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.186657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.186735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.186882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.186963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.187116 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.187197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.187349 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.187431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.187579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.187659 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.187812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.187901 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.188030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.188114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.188261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.188344 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.188507 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.188597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.188737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.188819 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.188968 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.189047 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.189219 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.189318 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.189477 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.189565 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.189710 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.189790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.189937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.190023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.190170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.190251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.190400 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.190486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.190633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.190713 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.190865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.190948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.191112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.191192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.191351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.191438 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.191598 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.191679 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.191830 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.191910 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.192062 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.192188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.192290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.192371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.192525 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.192605 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.192756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.192843 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.192995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.193096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.193254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.193334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.193488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.193581 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.193723 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.193805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.193955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.194035 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.194187 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.194277 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.194446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.194525 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.194681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.194770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.194917 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.195006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.195152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.195239 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.195388 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.195469 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.195621 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.195708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.195853 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.195934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.196085 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.196166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.196316 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.196404 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.196543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.196629 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.196799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.196871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.197175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.197325 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.197454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.197578 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.197703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.197828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.197952 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.198088 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.198217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.198343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.198468 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.198592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.198717 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.198861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.198987 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.199113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.199240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.199364 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.199489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.external = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.199621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] role.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.199746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.199872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] role.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.199872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.200107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.200107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.200353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.200489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.password = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.200621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.200747 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.token = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.200872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.200997 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.201158 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.201287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.201413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.201539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.201664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.201788 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.201920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.202045 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.202170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.202313 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.202439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.202565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.202697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.202823 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.202947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.203072 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.203203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.203338 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.203464 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.203589 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.203589 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.203824 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.203961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.204094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.204219 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.204351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.204477 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.204603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.204727 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.204860 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.204985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.205146 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.205271 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.205401 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.205401 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.205715 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.205859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.205984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.206114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.206241 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.206375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.206375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.206627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.206755 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.206881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.207007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.207133 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.207268 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.207395 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.207531 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.207659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.207785 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.207910 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.208046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.208175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.208300 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.208426 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.208555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.208682 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.208816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.208943 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.209100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.209236 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.209361 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.209486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.209688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.209817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.209941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.210072 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.210203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.210328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.210460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.210585 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.210710 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.210835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.210959 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.211094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.211094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.211339 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.211339 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.211573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.211573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.211807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.211941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.212065 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.212190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.212825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.213025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.213265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.213402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.213540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.213668 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.213794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.213920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.214059 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.connection = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.214195 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.214335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.214464 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.214590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.214718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.214843 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.214970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.215118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.215258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.215385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.215385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.215627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.215760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.215895 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.216022 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.216149 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.216149 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.216394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.216522 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.216656 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.216783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.216908 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.217045 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.217188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.217316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.217455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.217586 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.217714 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.217842 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.217970 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.218138 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.218280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.218415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.218546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.218678 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.218809 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.218809 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 15:20:10.219080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.219080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.219080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.219080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.219080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.219080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.219080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.219080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.219080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.220126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.220126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.220126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.220126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.220126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.220126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.220843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.220843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.220843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.220843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.221366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.221366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.221609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.221766 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.221996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.222239 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.222483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.222726 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.222958 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.223185 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.223417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.223649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.223882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.224206 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.224454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.224702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.224947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.225219 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.225467 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.225696 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.225941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.226176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.226410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.226642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.226882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.227124 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.227370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.227607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.227857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.228683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.229018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.229281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.229515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.229751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.229996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.230247 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.230476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.230709 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.230948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.231198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.231443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.231675 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.231922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.232170 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.232402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.232644 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.232878 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.233139 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.233375 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] role.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.233619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] role.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.233848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.234092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.234337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.234585 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.234829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.235076 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.235320 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.235565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.235809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.236041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.236285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.236530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.236779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.237025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.237440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.237676 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.237922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.238168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.238414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.238658 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.238890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.239149 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.239398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.239633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.239863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.240094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.240322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.240570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.240822 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.241088 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.241324 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.241559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.241786 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.242020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.242256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.242501 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.242734 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.242985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.243228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.243463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.244353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.244651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.244900 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.245155 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.245395 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.245629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.245873 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.246118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.246345 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.246584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.246877 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.247142 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.connection = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.247393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.247636 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.247870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.248107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.248236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.248377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.248506 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.248637 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.248764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.248896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.249031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.249256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.249491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.249732 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.249963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.250207 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.250440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.250680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.250908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.251146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.251376 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:20:10.251544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 15:20:10.257208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.276689 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.276937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.277171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.286275 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.286512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.287039 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.308712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.308937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.339858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.345590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.391304 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.391531 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.393198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.393438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.457387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.457633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.457779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.457915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.458049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.458183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.458324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.458455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.458584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.458724 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.458851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.458987 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.482140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.482376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.482513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.482644 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.482773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.482912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.483041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.483187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.483315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.483442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.483582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.483710 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.483837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.483837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.495281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.495643 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.507659 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.507890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.508034 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.508164 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.508305 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.508438 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.508563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.508690 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.508816 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.508941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.509105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.509235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.509788 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.509788 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:20:10.526502 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.526710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 15:20:10.527046 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.527174 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.527300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 15:20:10.527426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 15:20:10.527553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.527688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.527815 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.527940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 15:20:10.528077 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.528202 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.528336 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.528462 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.528632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.528892 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.539909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.540188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.540989 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.540989 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.541632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.541770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.541980 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.542259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.549372 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.550371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.552647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.552927 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.553235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.553522 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.553651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.553809 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.553949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.554075 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.554212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.554338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.554712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.554955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.555083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.555217 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.555344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.555493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.555631 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.555759 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.555893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.556288 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.556288 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.556830 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.583986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.585752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.585974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.586118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.588522 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.588795 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.591919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.592156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.592443 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.594101 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.594381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.594662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.599727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.600011 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.601911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.602192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.605107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.605520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.605814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.606103 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.606385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.606663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.606940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.607034 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.607213 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.607351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.607633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.607921 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.608200 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.608474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.608742 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.609011 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.626332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.626615 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.627983 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.628262 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.631270 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.632836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.635569 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.635863 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.637322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.637619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.641005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.642542 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.644614 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.646308 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.648381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.648693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.648983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.649304 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.649586 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.649966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.650089 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.650346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.650641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.650933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.651220 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.651506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.669003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.669306 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.669576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.669843 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.670352 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.670635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.670902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.671166 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.677807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.679119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.679779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.680058 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.681602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.682160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.683768 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.684053 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.684348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.684635 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.684911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.684996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.685213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.685493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.685773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.686048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.699010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.699295 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.699584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.699868 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.700014 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.700146 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.700299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.700422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.700581 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.700707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.700869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.701173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.701454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.701728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.718775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.719587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.722065 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.722822 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.725397 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.726128 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.727001 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.727721 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.728673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.728956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.729266 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.729591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.729873 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.730162 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.736643 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.737519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.738287 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.739121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.741544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.741842 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.742127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.742339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.742421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.742629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.742912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.743192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.752006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.752296 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.752678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.752882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.753195 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.753582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.761323 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.761607 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.761892 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.762039 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:20:10.762169 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.762318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.762441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.762607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.762698 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.762886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.762995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.763157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.763268 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.763424 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.763555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.763693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.763827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.763979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.764099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.764255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.764362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.764515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.764643 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.764782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.764911 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.765041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.765328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.765605 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:20:10.803581 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b7ae4a29c0 pid: 80598 (default app) May 23 15:20:10.804197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b7ae4a29c0 pid: 80599 (default app) May 23 15:20:10.807611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ca498-2088-4c4c-a097-884667948002 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:10.807907 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ca498-2088-4c4c-a097-884667948002 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:10.808188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ca498-2088-4c4c-a097-884667948002 None None] PATH_INFO: `/v3` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:10.812496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1/1] 10.222.0.64 () {50 vars in 791 bytes} [Mon May 23 15:20:10 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:19.611181 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5556a44-aa82-4bf7-b3f4-3d3a2531c74b None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:19.611698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5556a44-aa82-4bf7-b3f4-3d3a2531c74b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:19.611812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5556a44-aa82-4bf7-b3f4-3d3a2531c74b None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:19.616271 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1/2] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:19.622587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dc87f-8b6e-4c29-8d9d-0f4a6a7fdc1c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:19.623040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dc87f-8b6e-4c29-8d9d-0f4a6a7fdc1c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:19.623152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dc87f-8b6e-4c29-8d9d-0f4a6a7fdc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:19.692189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: INFO dbcounter [None req-b76dc87f-8b6e-4c29-8d9d-0f4a6a7fdc1c None None] Registered counter for database keystone May 23 15:20:19.714383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG oslo_db.sqlalchemy.engines [None req-b76dc87f-8b6e-4c29-8d9d-0f4a6a7fdc1c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80598) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:20:19.720745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writer thread running {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:20:19.898614 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG passlib.handlers.bcrypt [None req-b76dc87f-8b6e-4c29-8d9d-0f4a6a7fdc1c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80598) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 15:20:19.898964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG passlib.handlers.bcrypt [None req-b76dc87f-8b6e-4c29-8d9d-0f4a6a7fdc1c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80598) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 15:20:19.956586 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b76dc87f-8b6e-4c29-8d9d-0f4a6a7fdc1c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:19.958238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b76dc87f-8b6e-4c29-8d9d-0f4a6a7fdc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:20.084367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 2/3] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 23 15:20:20.255430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5556a44-aa82-4bf7-b3f4-3d3a2531c74b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:20.324057 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: INFO dbcounter [None req-f5556a44-aa82-4bf7-b3f4-3d3a2531c74b None None] Registered counter for database keystone May 23 15:20:20.343671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG oslo_db.sqlalchemy.engines [None req-f5556a44-aa82-4bf7-b3f4-3d3a2531c74b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80599) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:20:20.349745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writer thread running {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:20:20.468879 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5556a44-aa82-4bf7-b3f4-3d3a2531c74b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:20.473115 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee986247-dc3e-4bc3-9b06-06e5a68200a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:20.473778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee986247-dc3e-4bc3-9b06-06e5a68200a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:20.474065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee986247-dc3e-4bc3-9b06-06e5a68200a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:20.474346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee986247-dc3e-4bc3-9b06-06e5a68200a5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:20.482554 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee986247-dc3e-4bc3-9b06-06e5a68200a5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:20.533637 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ee986247-dc3e-4bc3-9b06-06e5a68200a5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 15:20:20.533988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee986247-dc3e-4bc3-9b06-06e5a68200a5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:20.543547 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 2/4] 10.222.0.64 () {58 vars in 1324 bytes} [Mon May 23 15:20:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:20:20.549599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dc87f-8b6e-4c29-8d9d-0f4a6a7fdc1c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:20.565105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dc87f-8b6e-4c29-8d9d-0f4a6a7fdc1c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:20.569560 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e30d1-d799-4dd3-8a92-d06396763a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:20.570686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e30d1-d799-4dd3-8a92-d06396763a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:20.570978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e30d1-d799-4dd3-8a92-d06396763a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:20.572321 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e30d1-d799-4dd3-8a92-d06396763a3f admin admin] PATH_INFO: `/v3/projects/4c9cfa8acbce47f6b0259cbfbf5b338d` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:20.573883 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e30d1-d799-4dd3-8a92-d06396763a3f admin admin] RBAC: Authorizing `identity:get_project(project_id=4c9cfa8acbce47f6b0259cbfbf5b338d)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:20.737872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d01e30d1-d799-4dd3-8a92-d06396763a3f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 15:20:20.738473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e30d1-d799-4dd3-8a92-d06396763a3f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:20.742948 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 3/5] 10.222.0.64 () {56 vars in 1207 bytes} [Mon May 23 15:20:20 2022] GET /identity/v3/projects/4c9cfa8acbce47f6b0259cbfbf5b338d => generated 344 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:22.925383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b993c-a19b-47de-9dd4-007f8abf3965 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:22.925896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b993c-a19b-47de-9dd4-007f8abf3965 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:22.926002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b993c-a19b-47de-9dd4-007f8abf3965 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:22.930332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 3/6] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:20:22.936715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754c8bf-5d81-4f6f-8ad6-3983e97e5599 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:22.937116 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754c8bf-5d81-4f6f-8ad6-3983e97e5599 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:22.937326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754c8bf-5d81-4f6f-8ad6-3983e97e5599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:22.956882 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f754c8bf-5d81-4f6f-8ad6-3983e97e5599 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:22.958392 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f754c8bf-5d81-4f6f-8ad6-3983e97e5599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:22.966551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 4/7] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 23 15:20:22.988674 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b993c-a19b-47de-9dd4-007f8abf3965 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:23.001185 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b993c-a19b-47de-9dd4-007f8abf3965 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:23.005371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea7df9-f376-4129-9c78-399df8884b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:23.006004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7df9-f376-4129-9c78-399df8884b59 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:23.006295 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7df9-f376-4129-9c78-399df8884b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:23.006582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7df9-f376-4129-9c78-399df8884b59 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:23.014852 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea7df9-f376-4129-9c78-399df8884b59 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:23.016649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea7df9-f376-4129-9c78-399df8884b59 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:23.025676 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 4/8] 10.222.0.64 () {58 vars in 1324 bytes} [Mon May 23 15:20:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:20:23.031065 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754c8bf-5d81-4f6f-8ad6-3983e97e5599 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:23.040524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754c8bf-5d81-4f6f-8ad6-3983e97e5599 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:23.044866 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09838e-1f33-4ee7-9003-4167008b4067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:23.045765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09838e-1f33-4ee7-9003-4167008b4067 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:23.046054 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09838e-1f33-4ee7-9003-4167008b4067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:23.046337 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09838e-1f33-4ee7-9003-4167008b4067 admin admin] PATH_INFO: `/v3/users/2781ce56481d4d179e62cc19ea41a8b0` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:23.047725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09838e-1f33-4ee7-9003-4167008b4067 admin admin] RBAC: Authorizing `identity:get_user(user_id=2781ce56481d4d179e62cc19ea41a8b0)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:23.049613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09838e-1f33-4ee7-9003-4167008b4067 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:23.051179 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 5/9] 10.222.0.64 () {56 vars in 1198 bytes} [Mon May 23 15:20:23 2022] GET /identity/v3/users/2781ce56481d4d179e62cc19ea41a8b0 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:25.302937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3b5ac-6f53-4776-a25f-8b1397c57d31 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:25.303458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3b5ac-6f53-4776-a25f-8b1397c57d31 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:25.303542 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3b5ac-6f53-4776-a25f-8b1397c57d31 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:25.305110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 5/10] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:25.310453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cb35e-0776-46fb-82a5-34caebd41c33 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:25.310863 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cb35e-0776-46fb-82a5-34caebd41c33 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:25.311038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cb35e-0776-46fb-82a5-34caebd41c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:25.316121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae1e82-2dd7-4a56-9208-3b7461c5167d None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:25.316673 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae1e82-2dd7-4a56-9208-3b7461c5167d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:25.316956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae1e82-2dd7-4a56-9208-3b7461c5167d None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:25.318263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 6/11] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:20:25.323864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf57b7-6823-4c01-9338-eb009b9f8edf None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:25.324153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf57b7-6823-4c01-9338-eb009b9f8edf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:25.324432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf57b7-6823-4c01-9338-eb009b9f8edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:25.328753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-354cb35e-0776-46fb-82a5-34caebd41c33 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:25.329996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-354cb35e-0776-46fb-82a5-34caebd41c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:25.364453 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG passlib.handlers.bcrypt [None req-3abf57b7-6823-4c01-9338-eb009b9f8edf None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80599) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 15:20:25.364870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG passlib.handlers.bcrypt [None req-3abf57b7-6823-4c01-9338-eb009b9f8edf None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80599) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 15:20:25.408982 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 6/12] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:25.411787 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3abf57b7-6823-4c01-9338-eb009b9f8edf None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:25.413379 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3abf57b7-6823-4c01-9338-eb009b9f8edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:25.432174 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cb35e-0776-46fb-82a5-34caebd41c33 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:25.456511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:25.511978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 7/13] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:20:25.534308 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf57b7-6823-4c01-9338-eb009b9f8edf None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:25.535504 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cb35e-0776-46fb-82a5-34caebd41c33 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:25.568713 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6a8bb-7a2a-4914-958d-e2091cb62b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:25.569481 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6a8bb-7a2a-4914-958d-e2091cb62b4d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:25.569770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6a8bb-7a2a-4914-958d-e2091cb62b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:25.570050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6a8bb-7a2a-4914-958d-e2091cb62b4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:25.581483 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6a8bb-7a2a-4914-958d-e2091cb62b4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c9b6a8bb-7a2a-4914-958d-e2091cb62b4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:25.592869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:25.593810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:25.594676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:25.594676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:25.594676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:25.594676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:25.594676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:25.594676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:25.594676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:25.594676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:25.594676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:25.594676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:25.594676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:25.594676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:20:25.594676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 7/14] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:20:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:20:25.599852 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6a8bb-7a2a-4914-958d-e2091cb62b4d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:25.636852 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf57b7-6823-4c01-9338-eb009b9f8edf None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:25.639139 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:25.673487 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde03a8-78fa-435c-9bf4-91e0125a3253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:25.674392 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde03a8-78fa-435c-9bf4-91e0125a3253 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:25.674939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde03a8-78fa-435c-9bf4-91e0125a3253 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:25.675280 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde03a8-78fa-435c-9bf4-91e0125a3253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:25.682438 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:25.705224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6a8bb-7a2a-4914-958d-e2091cb62b4d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:25.740722 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4c986-d5d3-4764-b2f3-dc1a20f5378e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:25.741432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4c986-d5d3-4764-b2f3-dc1a20f5378e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:25.741725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4c986-d5d3-4764-b2f3-dc1a20f5378e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:25.742006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4c986-d5d3-4764-b2f3-dc1a20f5378e None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:25.742759 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4c986-d5d3-4764-b2f3-dc1a20f5378e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:25.743073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4c986-d5d3-4764-b2f3-dc1a20f5378e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:25.744480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4c986-d5d3-4764-b2f3-dc1a20f5378e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:25.744766 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde03a8-78fa-435c-9bf4-91e0125a3253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:25.746316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde03a8-78fa-435c-9bf4-91e0125a3253 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:25.756413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 8/15] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:25.762006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4c986-d5d3-4764-b2f3-dc1a20f5378e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:25.843418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 8/16] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:20:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:20:25.848901 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde03a8-78fa-435c-9bf4-91e0125a3253 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:25.851607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:25.860433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4c986-d5d3-4764-b2f3-dc1a20f5378e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:25.893892 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0edd736-3e24-4577-86dc-c07d11eddb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:25.894767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0edd736-3e24-4577-86dc-c07d11eddb16 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:25.895051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0edd736-3e24-4577-86dc-c07d11eddb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:25.895358 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0edd736-3e24-4577-86dc-c07d11eddb16 None admin] PATH_INFO: `/v3/users/2781ce56481d4d179e62cc19ea41a8b0` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:25.896565 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0edd736-3e24-4577-86dc-c07d11eddb16 None admin] RBAC: Authorizing `identity:get_user(user_id=2781ce56481d4d179e62cc19ea41a8b0)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:25.898081 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0edd736-3e24-4577-86dc-c07d11eddb16 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:25.900325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 9/17] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:25 2022] GET /identity/v3/users/2781ce56481d4d179e62cc19ea41a8b0 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:25.905708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0edd736-3e24-4577-86dc-c07d11eddb16 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:25.920896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:25.949989 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde03a8-78fa-435c-9bf4-91e0125a3253 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:25.983822 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9433db-681b-4678-a9c5-b95def5f01dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:25.984618 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9433db-681b-4678-a9c5-b95def5f01dc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:25.984904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9433db-681b-4678-a9c5-b95def5f01dc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:25.985226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9433db-681b-4678-a9c5-b95def5f01dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:26.004038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9433db-681b-4678-a9c5-b95def5f01dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:26.005615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9433db-681b-4678-a9c5-b95def5f01dc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:26.006966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0edd736-3e24-4577-86dc-c07d11eddb16 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:26.027894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-dd9433db-681b-4678-a9c5-b95def5f01dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:20:26.027894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:26.027894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:26.027894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:26.027894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:26.027894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:26.027894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:26.027894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:26.027894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:26.027894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:26.027894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:26.027894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:26.027894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:20:26.027894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:20:26.027894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:26.028977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:26.030199 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:20:26.030199 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:20:26.030199 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:20:26.030199 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:20:26.030199 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 9/18] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:20:26.035158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9433db-681b-4678-a9c5-b95def5f01dc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:26.041689 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2c18d-de3a-4886-9f69-782011f83ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:26.042497 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2c18d-de3a-4886-9f69-782011f83ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:26.042784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2c18d-de3a-4886-9f69-782011f83ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:26.043062 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2c18d-de3a-4886-9f69-782011f83ad9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:26.044247 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2c18d-de3a-4886-9f69-782011f83ad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:26.045745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2c18d-de3a-4886-9f69-782011f83ad9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:26.047396 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 10/19] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:20:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:26.053045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2c18d-de3a-4886-9f69-782011f83ad9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:26.092771 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:26.137749 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9433db-681b-4678-a9c5-b95def5f01dc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:26.139678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:26.158652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2c18d-de3a-4886-9f69-782011f83ad9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:26.171622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb54749-4392-440a-bb6c-8ec819e7ceec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:26.172293 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb54749-4392-440a-bb6c-8ec819e7ceec None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:26.172588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb54749-4392-440a-bb6c-8ec819e7ceec None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:26.172877 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb54749-4392-440a-bb6c-8ec819e7ceec None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:26.173643 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb54749-4392-440a-bb6c-8ec819e7ceec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:26.173963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb54749-4392-440a-bb6c-8ec819e7ceec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:26.175359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb54749-4392-440a-bb6c-8ec819e7ceec None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:26.193678 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 10/20] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:26.193922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd38442-3c71-4bdb-b92a-62d6ded3a800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:26.194533 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd38442-3c71-4bdb-b92a-62d6ded3a800 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:26.194827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd38442-3c71-4bdb-b92a-62d6ded3a800 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:26.195110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd38442-3c71-4bdb-b92a-62d6ded3a800 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:26.195993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd38442-3c71-4bdb-b92a-62d6ded3a800 None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.domain.id=default, user.id=2781ce56481d4d179e62cc19ea41a8b0) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:26.196456 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd38442-3c71-4bdb-b92a-62d6ded3a800 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:26.197975 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd38442-3c71-4bdb-b92a-62d6ded3a800 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:26.205603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 11/21] 10.222.0.64 () {56 vars in 1322 bytes} [Mon May 23 15:20:26 2022] GET /identity/v3/role_assignments?user.id=2781ce56481d4d179e62cc19ea41a8b0&scope.domain.id=default&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:28.627040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a068625-4c3a-467e-b31d-a7fffa912932 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:28.628108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a068625-4c3a-467e-b31d-a7fffa912932 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:28.628663 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a068625-4c3a-467e-b31d-a7fffa912932 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:28.630264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 11/22] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:28.636551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848115e5-3365-4dd1-904f-b04be60113d9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:28.637053 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848115e5-3365-4dd1-904f-b04be60113d9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:28.637175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848115e5-3365-4dd1-904f-b04be60113d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:28.666385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-848115e5-3365-4dd1-904f-b04be60113d9 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:28.667636 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-848115e5-3365-4dd1-904f-b04be60113d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:28.776151 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 12/23] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:28.810790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a068625-4c3a-467e-b31d-a7fffa912932 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:28.986511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e22db3-3d5b-47bf-8efd-212e0fbb5306 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:28.987203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e22db3-3d5b-47bf-8efd-212e0fbb5306 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:28.987717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e22db3-3d5b-47bf-8efd-212e0fbb5306 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:28.993188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 13/24] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:28.995588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c007023-cd49-4c24-b67d-903e1dc6abfa None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:28.996104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c007023-cd49-4c24-b67d-903e1dc6abfa None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:28.996588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c007023-cd49-4c24-b67d-903e1dc6abfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:29.034661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-5c007023-cd49-4c24-b67d-903e1dc6abfa None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:29.036274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5c007023-cd49-4c24-b67d-903e1dc6abfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:29.117871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a068625-4c3a-467e-b31d-a7fffa912932 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:29.154985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:29.179620 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 14/25] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:29.187895 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09aefe-93a5-4466-8965-6c9b6bdf4361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:29.188840 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09aefe-93a5-4466-8965-6c9b6bdf4361 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:29.189155 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09aefe-93a5-4466-8965-6c9b6bdf4361 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:29.189440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09aefe-93a5-4466-8965-6c9b6bdf4361 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:29.212184 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09aefe-93a5-4466-8965-6c9b6bdf4361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:29.226318 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c007023-cd49-4c24-b67d-903e1dc6abfa None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-cd09aefe-93a5-4466-8965-6c9b6bdf4361 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:29.226829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:29.228270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:29.229054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:29.229054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:29.229054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:29.229054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:29.229054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:29.229054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:29.229054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:29.229054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:29.229054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:29.229054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:29.229054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:29.229054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:20:29.229054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 12/26] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:20:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:29.234277 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:29.234653 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09aefe-93a5-4466-8965-6c9b6bdf4361 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:29.392647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c007023-cd49-4c24-b67d-903e1dc6abfa None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:29.405121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09aefe-93a5-4466-8965-6c9b6bdf4361 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:29.428438 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6590d-e169-4853-86b7-7cde652542f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:29.429554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6590d-e169-4853-86b7-7cde652542f3 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:29.429848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6590d-e169-4853-86b7-7cde652542f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:29.430675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6590d-e169-4853-86b7-7cde652542f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:29.441530 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6590d-e169-4853-86b7-7cde652542f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:29.443369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6590d-e169-4853-86b7-7cde652542f3 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:29.445388 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6895b3e-89f8-4250-93f3-cf50c5dd8562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:29.446235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6895b3e-89f8-4250-93f3-cf50c5dd8562 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:29.446710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6895b3e-89f8-4250-93f3-cf50c5dd8562 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:29.447005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6895b3e-89f8-4250-93f3-cf50c5dd8562 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:29.447769 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6895b3e-89f8-4250-93f3-cf50c5dd8562 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:29.448085 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6895b3e-89f8-4250-93f3-cf50c5dd8562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:29.449512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6895b3e-89f8-4250-93f3-cf50c5dd8562 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:29.463050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 13/27] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:29.463454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-26d6590d-e169-4853-86b7-7cde652542f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:20:29.463454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:29.463454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:29.463454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:29.463454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:29.463454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:29.463454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:29.463454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:29.463454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:29.463454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:29.463454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:29.463454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:29.463454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:20:29.463454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:20:29.463454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:29.464238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:29.465072 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:20:29.465072 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:20:29.465072 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:20:29.465072 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:20:29.465072 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 15/28] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:20:29.468248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a44c8-7a59-42eb-9f5b-6354cc42172d None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:29.468248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a44c8-7a59-42eb-9f5b-6354cc42172d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:29.468248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a44c8-7a59-42eb-9f5b-6354cc42172d None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:29.468248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 16/29] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:29.469592 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4c85e-f4da-49e3-a265-f6973eb3991d None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:29.469893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4c85e-f4da-49e3-a265-f6973eb3991d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:29.470183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4c85e-f4da-49e3-a265-f6973eb3991d None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:29.471460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 14/30] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:29.474550 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a44c8-7a59-42eb-9f5b-6354cc42172d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:29.476229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4c85e-f4da-49e3-a265-f6973eb3991d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:29.522182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:29.539475 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:29.584689 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a44c8-7a59-42eb-9f5b-6354cc42172d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:29.590350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4c85e-f4da-49e3-a265-f6973eb3991d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:29.621431 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4cc14-d40b-48a8-b3cb-fbecd1faaf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:29.622327 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4cc14-d40b-48a8-b3cb-fbecd1faaf9b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:29.622611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4cc14-d40b-48a8-b3cb-fbecd1faaf9b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:29.622900 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4cc14-d40b-48a8-b3cb-fbecd1faaf9b None admin] PATH_INFO: `/v3/users/2781ce56481d4d179e62cc19ea41a8b0` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:29.624200 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4cc14-d40b-48a8-b3cb-fbecd1faaf9b None admin] RBAC: Authorizing `identity:get_user(user_id=2781ce56481d4d179e62cc19ea41a8b0)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:29.625899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4cc14-d40b-48a8-b3cb-fbecd1faaf9b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:29.628166 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d6e26-8365-463a-acb3-5832a18166d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:29.628914 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d6e26-8365-463a-acb3-5832a18166d9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:29.629255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d6e26-8365-463a-acb3-5832a18166d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:29.629540 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d6e26-8365-463a-acb3-5832a18166d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:29.630271 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d6e26-8365-463a-acb3-5832a18166d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:29.630585 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d6e26-8365-463a-acb3-5832a18166d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:29.632051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d6e26-8365-463a-acb3-5832a18166d9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:29.632441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 17/31] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:29 2022] GET /identity/v3/users/2781ce56481d4d179e62cc19ea41a8b0 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:29.634300 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88566d4d-f419-4a5f-885d-d6baf589e622 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:29.634598 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88566d4d-f419-4a5f-885d-d6baf589e622 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:29.634882 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88566d4d-f419-4a5f-885d-d6baf589e622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:29.645451 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 15/32] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:29.646774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55b307-554e-43e4-9286-46dc1b17376e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:29.647074 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55b307-554e-43e4-9286-46dc1b17376e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:29.647359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55b307-554e-43e4-9286-46dc1b17376e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:29.656874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-88566d4d-f419-4a5f-885d-d6baf589e622 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:29.658119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-88566d4d-f419-4a5f-885d-d6baf589e622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:29.666492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4b55b307-554e-43e4-9286-46dc1b17376e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:29.667744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4b55b307-554e-43e4-9286-46dc1b17376e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:29.733413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 18/33] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:29.734328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88566d4d-f419-4a5f-885d-d6baf589e622 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:29.763981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 16/34] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:29.765174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55b307-554e-43e4-9286-46dc1b17376e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:29.796749 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:29.809661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:29.855874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88566d4d-f419-4a5f-885d-d6baf589e622 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:29.877989 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55b307-554e-43e4-9286-46dc1b17376e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:29.891246 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a1366-89d0-425e-9692-ee55573ea902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:29.892070 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a1366-89d0-425e-9692-ee55573ea902 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:29.892486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a1366-89d0-425e-9692-ee55573ea902 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:29.892619 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a1366-89d0-425e-9692-ee55573ea902 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:29.893893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a1366-89d0-425e-9692-ee55573ea902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:29.895387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a1366-89d0-425e-9692-ee55573ea902 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:29.897103 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 19/35] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:20:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:29.897811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a1366-89d0-425e-9692-ee55573ea902 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:29.920260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8422c18-d093-4f71-a833-a0322ac54242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:29.921295 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8422c18-d093-4f71-a833-a0322ac54242 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:29.921838 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8422c18-d093-4f71-a833-a0322ac54242 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:29.922269 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8422c18-d093-4f71-a833-a0322ac54242 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:29.923440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8422c18-d093-4f71-a833-a0322ac54242 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:29.925154 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8422c18-d093-4f71-a833-a0322ac54242 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:30.008303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a1366-89d0-425e-9692-ee55573ea902 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:30.035792 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:30.051367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd46dd-5a10-4d30-9fe6-5187622d8daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:30.052489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd46dd-5a10-4d30-9fe6-5187622d8daa None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:30.053374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd46dd-5a10-4d30-9fe6-5187622d8daa None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:30.054478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd46dd-5a10-4d30-9fe6-5187622d8daa None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:30.056249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd46dd-5a10-4d30-9fe6-5187622d8daa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:30.059289 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd46dd-5a10-4d30-9fe6-5187622d8daa None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:30.137095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 17/36] 10.222.0.64 () {60 vars in 1141 bytes} [Mon May 23 15:20:29 2022] POST /identity/v3/projects => generated 301 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:20:30.137095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8422c18-d093-4f71-a833-a0322ac54242 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:30.227920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:30.249962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8422c18-d093-4f71-a833-a0322ac54242 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:30.295102 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70efd6-d114-43e9-b996-f29e2667e959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:30.296082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70efd6-d114-43e9-b996-f29e2667e959 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:30.296617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70efd6-d114-43e9-b996-f29e2667e959 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:30.297267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70efd6-d114-43e9-b996-f29e2667e959 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:30.298214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70efd6-d114-43e9-b996-f29e2667e959 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:30.299932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70efd6-d114-43e9-b996-f29e2667e959 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:30.324832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 20/37] 10.222.0.64 () {60 vars in 1131 bytes} [Mon May 23 15:20:29 2022] POST /identity/v3/roles => generated 225 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:20:30.326487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd46dd-5a10-4d30-9fe6-5187622d8daa None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:30.370238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 18/38] 10.222.0.64 () {60 vars in 1131 bytes} [Mon May 23 15:20:30 2022] POST /identity/v3/roles => generated 219 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:20:30.465361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c180216-6c69-4275-b207-fc965b75a005 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:30.465991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c180216-6c69-4275-b207-fc965b75a005 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:30.466467 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c180216-6c69-4275-b207-fc965b75a005 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:30.468215 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 19/39] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:20:30.474224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2e3bb-f9c8-4968-8637-c970c816a060 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:30.474797 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2e3bb-f9c8-4968-8637-c970c816a060 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:30.475279 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2e3bb-f9c8-4968-8637-c970c816a060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:30.502973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-abd2e3bb-f9c8-4968-8637-c970c816a060 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:30.507173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-abd2e3bb-f9c8-4968-8637-c970c816a060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:30.523648 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd46dd-5a10-4d30-9fe6-5187622d8daa None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:30.593432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac64806-d97e-43ff-96de-1cce71c292dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:30.594992 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac64806-d97e-43ff-96de-1cce71c292dd None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:30.595470 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac64806-d97e-43ff-96de-1cce71c292dd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:30.596487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac64806-d97e-43ff-96de-1cce71c292dd None admin] PATH_INFO: `/v3/domains/default/users/2781ce56481d4d179e62cc19ea41a8b0/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:30.599942 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac64806-d97e-43ff-96de-1cce71c292dd None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2781ce56481d4d179e62cc19ea41a8b0, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:30.603312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac64806-d97e-43ff-96de-1cce71c292dd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:30.612388 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 20/40] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:20:30.614957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7c4c2-db95-4785-bde5-c751fc49754a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:30.614957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7c4c2-db95-4785-bde5-c751fc49754a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:30.614957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7c4c2-db95-4785-bde5-c751fc49754a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:30.616272 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 21/41] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:30.622634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf3ecf-5fa7-464b-a2b1-ab69f02ac1c3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:30.622935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf3ecf-5fa7-464b-a2b1-ab69f02ac1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:30.623221 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf3ecf-5fa7-464b-a2b1-ab69f02ac1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:30.644883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-edcf3ecf-5fa7-464b-a2b1-ab69f02ac1c3 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:30.646154 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-edcf3ecf-5fa7-464b-a2b1-ab69f02ac1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:30.710526 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 21/42] 10.222.0.64 () {58 vars in 1361 bytes} [Mon May 23 15:20:30 2022] PUT /identity/v3/domains/default/users/2781ce56481d4d179e62cc19ea41a8b0/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:20:30.711306 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac64806-d97e-43ff-96de-1cce71c292dd None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:30.726161 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:30.734433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 22/43] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:30.761093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf3ecf-5fa7-464b-a2b1-ab69f02ac1c3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:30.779878 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:30.863261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac64806-d97e-43ff-96de-1cce71c292dd None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:30.881127 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf3ecf-5fa7-464b-a2b1-ab69f02ac1c3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:30.910442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265c04f-5958-45bc-88ee-b638eb2a1352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:30.911263 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265c04f-5958-45bc-88ee-b638eb2a1352 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:30.911554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265c04f-5958-45bc-88ee-b638eb2a1352 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:30.911831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265c04f-5958-45bc-88ee-b638eb2a1352 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:30.914305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265c04f-5958-45bc-88ee-b638eb2a1352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:30.916053 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265c04f-5958-45bc-88ee-b638eb2a1352 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:30.917943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 22/44] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:20:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:30.921346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91243b0f-98fe-4332-9672-2506fde1f8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:30.922359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91243b0f-98fe-4332-9672-2506fde1f8c9 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:30.922462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91243b0f-98fe-4332-9672-2506fde1f8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:30.922672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91243b0f-98fe-4332-9672-2506fde1f8c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:30.923173 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265c04f-5958-45bc-88ee-b638eb2a1352 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:30.923441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91243b0f-98fe-4332-9672-2506fde1f8c9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:30.924847 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91243b0f-98fe-4332-9672-2506fde1f8c9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:30.996576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 23/45] 10.222.0.64 () {60 vars in 1131 bytes} [Mon May 23 15:20:30 2022] POST /identity/v3/roles => generated 223 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:20:31.036689 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265c04f-5958-45bc-88ee-b638eb2a1352 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:31.050307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:31.076836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf190f0-c16c-4456-a5c3-9f15e9dabb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:31.077581 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf190f0-c16c-4456-a5c3-9f15e9dabb43 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:31.077883 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf190f0-c16c-4456-a5c3-9f15e9dabb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:31.078175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf190f0-c16c-4456-a5c3-9f15e9dabb43 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:31.078916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf190f0-c16c-4456-a5c3-9f15e9dabb43 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:31.080509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf190f0-c16c-4456-a5c3-9f15e9dabb43 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:31.162977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 23/46] 10.222.0.64 () {60 vars in 1141 bytes} [Mon May 23 15:20:30 2022] POST /identity/v3/projects => generated 312 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:20:32.941923 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd6bf8-d3ac-4f9b-bdcc-8b984c88d00c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:32.942466 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd6bf8-d3ac-4f9b-bdcc-8b984c88d00c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:32.942552 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd6bf8-d3ac-4f9b-bdcc-8b984c88d00c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:32.943875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 24/47] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:32.949403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971f85e-a86b-4470-b8f1-0f7148810525 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:32.949818 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971f85e-a86b-4470-b8f1-0f7148810525 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:32.949912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971f85e-a86b-4470-b8f1-0f7148810525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:32.967127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c971f85e-a86b-4470-b8f1-0f7148810525 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:32.968645 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c971f85e-a86b-4470-b8f1-0f7148810525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:33.034111 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 24/48] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:33.058290 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd6bf8-d3ac-4f9b-bdcc-8b984c88d00c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:33.118786 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:33.161974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd6bf8-d3ac-4f9b-bdcc-8b984c88d00c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:33.198820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4687791-0cb9-425d-8232-04c09e460fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:33.199679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4687791-0cb9-425d-8232-04c09e460fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:33.199925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4687791-0cb9-425d-8232-04c09e460fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:33.200209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4687791-0cb9-425d-8232-04c09e460fa4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:33.210282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4687791-0cb9-425d-8232-04c09e460fa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f4687791-0cb9-425d-8232-04c09e460fa4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:33.219178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:33.220094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:33.221105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:33.221105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:33.221105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:33.221105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:33.221105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:33.221105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:33.221105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:33.221105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:33.221105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:33.221105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:33.221105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:33.221105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:20:33.221105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 25/49] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:20:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:33.226149 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971f85e-a86b-4470-b8f1-0f7148810525 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:33.322120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:33.331063 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971f85e-a86b-4470-b8f1-0f7148810525 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:33.354942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e980ac-cd64-422b-b17f-d4bc7e3cf74c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:33.355370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e980ac-cd64-422b-b17f-d4bc7e3cf74c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:33.355597 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e980ac-cd64-422b-b17f-d4bc7e3cf74c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:33.356986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 26/50] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:20:33.362198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ad6a0-6353-4c77-ba94-843d7f825ac4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:33.368126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ad6a0-6353-4c77-ba94-843d7f825ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:33.368126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ad6a0-6353-4c77-ba94-843d7f825ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:33.376566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da5ef3-5d1b-4eba-8e1f-79b629cd4d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:33.377320 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da5ef3-5d1b-4eba-8e1f-79b629cd4d5f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:33.377624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da5ef3-5d1b-4eba-8e1f-79b629cd4d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:33.377902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da5ef3-5d1b-4eba-8e1f-79b629cd4d5f None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:33.378668 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da5ef3-5d1b-4eba-8e1f-79b629cd4d5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:33.378985 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da5ef3-5d1b-4eba-8e1f-79b629cd4d5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:33.380191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f45ad6a0-6353-4c77-ba94-843d7f825ac4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:33.380463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da5ef3-5d1b-4eba-8e1f-79b629cd4d5f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:33.381461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f45ad6a0-6353-4c77-ba94-843d7f825ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:33.390011 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 25/51] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:33.395730 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da5ef3-5d1b-4eba-8e1f-79b629cd4d5f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:33.450122 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 27/52] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:20:33.473397 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ad6a0-6353-4c77-ba94-843d7f825ac4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:33.496817 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da5ef3-5d1b-4eba-8e1f-79b629cd4d5f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:33.522428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:33.530469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffe584-fa2a-4982-8edf-ff2e0d5a8efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:33.531380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffe584-fa2a-4982-8edf-ff2e0d5a8efd None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:33.536175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffe584-fa2a-4982-8edf-ff2e0d5a8efd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:33.536175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffe584-fa2a-4982-8edf-ff2e0d5a8efd None admin] PATH_INFO: `/v3/users/2781ce56481d4d179e62cc19ea41a8b0` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:33.536175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ffe584-fa2a-4982-8edf-ff2e0d5a8efd None admin] RBAC: Authorizing `identity:get_user(user_id=2781ce56481d4d179e62cc19ea41a8b0)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:33.536175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ffe584-fa2a-4982-8edf-ff2e0d5a8efd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:33.540821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 26/53] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:33 2022] GET /identity/v3/users/2781ce56481d4d179e62cc19ea41a8b0 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:33.541765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffe584-fa2a-4982-8edf-ff2e0d5a8efd None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:33.565805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:33.572689 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ad6a0-6353-4c77-ba94-843d7f825ac4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:33.606687 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7448507-8601-44a5-a7d0-b9f910a49291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:33.607491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7448507-8601-44a5-a7d0-b9f910a49291 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:33.607785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7448507-8601-44a5-a7d0-b9f910a49291 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:33.608070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7448507-8601-44a5-a7d0-b9f910a49291 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:33.609287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7448507-8601-44a5-a7d0-b9f910a49291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:33.610745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7448507-8601-44a5-a7d0-b9f910a49291 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:33.612365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 28/54] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:20:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:33.617886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7448507-8601-44a5-a7d0-b9f910a49291 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:33.641897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffe584-fa2a-4982-8edf-ff2e0d5a8efd None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:33.674841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f298a56-9360-4539-a34c-857830982784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:33.675634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f298a56-9360-4539-a34c-857830982784 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:33.675921 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f298a56-9360-4539-a34c-857830982784 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:33.676198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f298a56-9360-4539-a34c-857830982784 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:33.677399 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f298a56-9360-4539-a34c-857830982784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:33.678843 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f298a56-9360-4539-a34c-857830982784 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:33.680864 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 27/55] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:20:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:33.686630 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f298a56-9360-4539-a34c-857830982784 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:33.727978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7448507-8601-44a5-a7d0-b9f910a49291 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:33.744810 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:33.761271 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9406bb-9ec9-4f02-965d-b846d0b2c884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:33.761937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9406bb-9ec9-4f02-965d-b846d0b2c884 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:33.762226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9406bb-9ec9-4f02-965d-b846d0b2c884 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:33.762507 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9406bb-9ec9-4f02-965d-b846d0b2c884 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:33.763211 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9406bb-9ec9-4f02-965d-b846d0b2c884 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:33.764665 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9406bb-9ec9-4f02-965d-b846d0b2c884 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:33.782314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:33.791758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f298a56-9360-4539-a34c-857830982784 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:33.825398 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331615bc-5034-4fb1-924c-d7c75bd51094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:33.826058 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331615bc-5034-4fb1-924c-d7c75bd51094 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:33.826350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331615bc-5034-4fb1-924c-d7c75bd51094 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:33.826633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331615bc-5034-4fb1-924c-d7c75bd51094 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:33.827518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331615bc-5034-4fb1-924c-d7c75bd51094 None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.domain.id=default, user.id=2781ce56481d4d179e62cc19ea41a8b0) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:33.827881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331615bc-5034-4fb1-924c-d7c75bd51094 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:33.829351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331615bc-5034-4fb1-924c-d7c75bd51094 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:33.835241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 28/56] 10.222.0.64 () {56 vars in 1322 bytes} [Mon May 23 15:20:33 2022] GET /identity/v3/role_assignments?user.id=2781ce56481d4d179e62cc19ea41a8b0&scope.domain.id=default&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:33.842315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 29/57] 10.222.0.64 () {60 vars in 1140 bytes} [Mon May 23 15:20:33 2022] POST /identity/v3/projects => generated 298 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:20:36.068521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87180b8e-40fc-4779-967a-96d4a93522ab None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:36.069038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87180b8e-40fc-4779-967a-96d4a93522ab None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:36.069168 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87180b8e-40fc-4779-967a-96d4a93522ab None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:36.071001 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 29/58] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:36.076294 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58179fb3-ed7d-44e1-abfb-e5bb3462d91c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:36.076672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58179fb3-ed7d-44e1-abfb-e5bb3462d91c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:36.076872 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58179fb3-ed7d-44e1-abfb-e5bb3462d91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:36.094577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-58179fb3-ed7d-44e1-abfb-e5bb3462d91c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:36.095788 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-58179fb3-ed7d-44e1-abfb-e5bb3462d91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:36.160237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 30/59] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:36.184741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87180b8e-40fc-4779-967a-96d4a93522ab None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:36.282920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87180b8e-40fc-4779-967a-96d4a93522ab None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:36.316241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb9d553-82da-4d03-a4ab-805b60fba047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:36.317025 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9d553-82da-4d03-a4ab-805b60fba047 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:36.317344 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9d553-82da-4d03-a4ab-805b60fba047 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:36.317625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9d553-82da-4d03-a4ab-805b60fba047 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:36.318842 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb9d553-82da-4d03-a4ab-805b60fba047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:36.320532 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb9d553-82da-4d03-a4ab-805b60fba047 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:36.322179 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 30/60] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:20:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:36.327600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58179fb3-ed7d-44e1-abfb-e5bb3462d91c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:36.420388 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:36.423061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58179fb3-ed7d-44e1-abfb-e5bb3462d91c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:36.454997 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313e9e8-2357-4c3f-8b29-7def7aac445e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:36.455660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313e9e8-2357-4c3f-8b29-7def7aac445e None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:36.455949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313e9e8-2357-4c3f-8b29-7def7aac445e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:36.456229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313e9e8-2357-4c3f-8b29-7def7aac445e None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:36.456940 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313e9e8-2357-4c3f-8b29-7def7aac445e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:36.458414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313e9e8-2357-4c3f-8b29-7def7aac445e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:36.577606 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 31/61] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:20:36 2022] POST /identity/v3/users => generated 275 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:20:39.886935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b86b1b-f621-4b03-8e5f-0250917bf4d4 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:39.887923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b86b1b-f621-4b03-8e5f-0250917bf4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:39.888425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b86b1b-f621-4b03-8e5f-0250917bf4d4 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:39.890033 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 31/62] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:39.895503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d102f-8039-4cc5-a59d-0a84f6ef1991 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:39.896334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d102f-8039-4cc5-a59d-0a84f6ef1991 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:39.896813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d102f-8039-4cc5-a59d-0a84f6ef1991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:39.919619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fb8d102f-8039-4cc5-a59d-0a84f6ef1991 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:39.921238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fb8d102f-8039-4cc5-a59d-0a84f6ef1991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:39.991315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d270d5-c477-4bf7-bebc-fc85ab6fb100 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:39.992732 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d270d5-c477-4bf7-bebc-fc85ab6fb100 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:39.993243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d270d5-c477-4bf7-bebc-fc85ab6fb100 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:39.994897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 32/63] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:20:40.000026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e14ba-08cb-417a-93fa-f1155375b026 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:40.000663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e14ba-08cb-417a-93fa-f1155375b026 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:40.001160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e14ba-08cb-417a-93fa-f1155375b026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:40.025290 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 32/64] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:40.034080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-cf4e14ba-08cb-417a-93fa-f1155375b026 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:40.035362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cf4e14ba-08cb-417a-93fa-f1155375b026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:40.050533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d102f-8039-4cc5-a59d-0a84f6ef1991 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:40.089244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:40.141565 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 33/65] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:20:40.194625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:40.211962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d102f-8039-4cc5-a59d-0a84f6ef1991 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:40.238771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e14ba-08cb-417a-93fa-f1155375b026 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:40.293221 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b6b37-1f48-49d5-9f88-dca5d965f5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:40.294045 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b6b37-1f48-49d5-9f88-dca5d965f5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:40.294337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b6b37-1f48-49d5-9f88-dca5d965f5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:40.294620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b6b37-1f48-49d5-9f88-dca5d965f5d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:40.318562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b6b37-1f48-49d5-9f88-dca5d965f5d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-5c0b6b37-1f48-49d5-9f88-dca5d965f5d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:40.330833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:40.331725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:40.332512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:40.332512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:40.332512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:40.332512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:40.332512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:40.332512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:40.332512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:40.332512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:40.332512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:40.332512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:40.332512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:20:40.332512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:20:40.332512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 33/66] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:20:40.339762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b6b37-1f48-49d5-9f88-dca5d965f5d3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:40.407573 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e14ba-08cb-417a-93fa-f1155375b026 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:40.442625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd8754e-a902-449f-965f-68217c0d55ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:40.443423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8754e-a902-449f-965f-68217c0d55ec None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:40.443708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8754e-a902-449f-965f-68217c0d55ec None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:40.443993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8754e-a902-449f-965f-68217c0d55ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:40.454924 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd8754e-a902-449f-965f-68217c0d55ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-dfd8754e-a902-449f-965f-68217c0d55ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:40.463094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:40.463966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:40.464746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:40.464746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:40.464746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:40.464746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:40.464746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:40.464746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:40.464746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:40.464746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:40.464746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:40.464746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:40.464746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:40.464746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:20:40.465370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b6b37-1f48-49d5-9f88-dca5d965f5d3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:40.465440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 34/67] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:20:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:40.467033 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a070ea-fe64-40c8-99bd-37f83d567c40 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:40.467333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a070ea-fe64-40c8-99bd-37f83d567c40 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:40.467622 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a070ea-fe64-40c8-99bd-37f83d567c40 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:40.468968 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 35/68] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:40.470086 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965475de-0394-4069-8af8-859ffa687713 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:40.470440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965475de-0394-4069-8af8-859ffa687713 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:40.470724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965475de-0394-4069-8af8-859ffa687713 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:40.472079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 36/69] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:40.473097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037e495-374b-4ccf-acf7-39821dc0eddd None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:40.473455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037e495-374b-4ccf-acf7-39821dc0eddd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:40.473741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037e495-374b-4ccf-acf7-39821dc0eddd None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:40.475087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 37/70] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:40.477860 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037e495-374b-4ccf-acf7-39821dc0eddd None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:40.492866 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:40.499626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215340a8-45e7-41ea-a7e0-8456dc3414e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:40.500311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215340a8-45e7-41ea-a7e0-8456dc3414e5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:40.500602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215340a8-45e7-41ea-a7e0-8456dc3414e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:40.500885 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215340a8-45e7-41ea-a7e0-8456dc3414e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:40.501677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215340a8-45e7-41ea-a7e0-8456dc3414e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:40.505271 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215340a8-45e7-41ea-a7e0-8456dc3414e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:40.505271 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215340a8-45e7-41ea-a7e0-8456dc3414e5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:40.512380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 34/71] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:20:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:40.513683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5db1e-7197-4b0d-883f-1b478da070a0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:40.513981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5db1e-7197-4b0d-883f-1b478da070a0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:40.514265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5db1e-7197-4b0d-883f-1b478da070a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:40.530804 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-02d5db1e-7197-4b0d-883f-1b478da070a0 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:40.531999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-02d5db1e-7197-4b0d-883f-1b478da070a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:40.533961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:40.582865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037e495-374b-4ccf-acf7-39821dc0eddd None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:40.599959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 35/72] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:40.601299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353ce5a-13dd-4510-9dd5-47d06d06f98c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:40.601598 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353ce5a-13dd-4510-9dd5-47d06d06f98c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:40.601963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353ce5a-13dd-4510-9dd5-47d06d06f98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:40.616817 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7026d81-ddf6-495b-bfcd-76169260fbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:40.618254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7026d81-ddf6-495b-bfcd-76169260fbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:40.618515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7026d81-ddf6-495b-bfcd-76169260fbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:40.618849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7026d81-ddf6-495b-bfcd-76169260fbf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:40.619135 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f353ce5a-13dd-4510-9dd5-47d06d06f98c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:40.619605 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7026d81-ddf6-495b-bfcd-76169260fbf4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:40.619918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7026d81-ddf6-495b-bfcd-76169260fbf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:40.620374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f353ce5a-13dd-4510-9dd5-47d06d06f98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:40.621338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7026d81-ddf6-495b-bfcd-76169260fbf4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:40.636332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 38/73] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:40.636332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c80421-0870-473d-8bf6-b65acf88b4f1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:40.636332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c80421-0870-473d-8bf6-b65acf88b4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:40.636332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c80421-0870-473d-8bf6-b65acf88b4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:40.652061 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-68c80421-0870-473d-8bf6-b65acf88b4f1 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:40.653288 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-68c80421-0870-473d-8bf6-b65acf88b4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:40.686898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 36/74] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:40.701240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353ce5a-13dd-4510-9dd5-47d06d06f98c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:40.734626 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 39/75] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:40.735487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c80421-0870-473d-8bf6-b65acf88b4f1 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:40.738455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:40.761897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:40.794100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353ce5a-13dd-4510-9dd5-47d06d06f98c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:40.828799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42be4d-2a4a-4f75-a9df-fcc8a4009e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:40.829718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42be4d-2a4a-4f75-a9df-fcc8a4009e6f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:40.830168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42be4d-2a4a-4f75-a9df-fcc8a4009e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:40.830895 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42be4d-2a4a-4f75-a9df-fcc8a4009e6f None admin] PATH_INFO: `/v3/users/358e903ec0d14cd2809ef65ffb8035f5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:40.833400 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c80421-0870-473d-8bf6-b65acf88b4f1 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:40.846068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42be4d-2a4a-4f75-a9df-fcc8a4009e6f None admin] RBAC: Authorizing `identity:get_user(user_id=358e903ec0d14cd2809ef65ffb8035f5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:40.847535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42be4d-2a4a-4f75-a9df-fcc8a4009e6f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:40.849235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 37/76] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:40 2022] GET /identity/v3/users/358e903ec0d14cd2809ef65ffb8035f5 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:40.850086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42be4d-2a4a-4f75-a9df-fcc8a4009e6f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:40.871136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315550b5-760a-4d0b-a999-f6d7fb8112e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:40.871958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315550b5-760a-4d0b-a999-f6d7fb8112e5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:40.872247 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315550b5-760a-4d0b-a999-f6d7fb8112e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:40.872530 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315550b5-760a-4d0b-a999-f6d7fb8112e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:40.882531 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315550b5-760a-4d0b-a999-f6d7fb8112e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-315550b5-760a-4d0b-a999-f6d7fb8112e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:40.890677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:40.892265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:40.893249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:40.893249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:40.893249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:40.893249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:40.893249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:40.893249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:40.893249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:40.893249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:40.893249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:40.893249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:40.893249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:20:40.893249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:20:40.893249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 40/77] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:20:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:40.894884 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315550b5-760a-4d0b-a999-f6d7fb8112e5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:40.951841 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42be4d-2a4a-4f75-a9df-fcc8a4009e6f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:40.957537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:40.977223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:40.985683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd1e5b-b477-4744-a4b9-5f2e1d6dd6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:40.986565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd1e5b-b477-4744-a4b9-5f2e1d6dd6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:40.986853 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd1e5b-b477-4744-a4b9-5f2e1d6dd6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:40.987135 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd1e5b-b477-4744-a4b9-5f2e1d6dd6e6 None admin] PATH_INFO: `/v3/users/2781ce56481d4d179e62cc19ea41a8b0` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:40.988318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd1e5b-b477-4744-a4b9-5f2e1d6dd6e6 None admin] RBAC: Authorizing `identity:get_user(user_id=2781ce56481d4d179e62cc19ea41a8b0)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:40.989842 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd1e5b-b477-4744-a4b9-5f2e1d6dd6e6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:40.991469 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 38/78] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:40 2022] GET /identity/v3/users/2781ce56481d4d179e62cc19ea41a8b0 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:40.996500 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd1e5b-b477-4744-a4b9-5f2e1d6dd6e6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:41.000309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315550b5-760a-4d0b-a999-f6d7fb8112e5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:41.034458 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88645390-239f-4e40-9ef2-6c8ca86e6d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:41.035261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88645390-239f-4e40-9ef2-6c8ca86e6d3e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:41.035555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88645390-239f-4e40-9ef2-6c8ca86e6d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:41.035842 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88645390-239f-4e40-9ef2-6c8ca86e6d3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:41.037044 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88645390-239f-4e40-9ef2-6c8ca86e6d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:41.038546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88645390-239f-4e40-9ef2-6c8ca86e6d3e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:41.040328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 41/79] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:20:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:41.041135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88645390-239f-4e40-9ef2-6c8ca86e6d3e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:41.092970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd1e5b-b477-4744-a4b9-5f2e1d6dd6e6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:41.125984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1f33e-1e90-46ca-a44a-f5898e9b229b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:41.126777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1f33e-1e90-46ca-a44a-f5898e9b229b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:41.127063 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1f33e-1e90-46ca-a44a-f5898e9b229b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:41.127339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1f33e-1e90-46ca-a44a-f5898e9b229b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:41.138240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1f33e-1e90-46ca-a44a-f5898e9b229b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:41.143002 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88645390-239f-4e40-9ef2-6c8ca86e6d3e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-19e1f33e-1e90-46ca-a44a-f5898e9b229b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:41.145979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:41.147052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:41.147856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:41.147856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:41.147856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:41.147856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:41.147856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:41.147856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:41.147856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:41.147856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:41.147856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:41.147856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:41.147856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:20:41.147856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:20:41.147856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 39/80] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:41.149143 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1f33e-1e90-46ca-a44a-f5898e9b229b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:41.173280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:41.178251 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bca02c-fadb-4d7e-86d3-719ef8fd68a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:41.179167 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bca02c-fadb-4d7e-86d3-719ef8fd68a1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:41.179641 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bca02c-fadb-4d7e-86d3-719ef8fd68a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:41.179930 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bca02c-fadb-4d7e-86d3-719ef8fd68a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:41.208748 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:41.249912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bca02c-fadb-4d7e-86d3-719ef8fd68a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:41.251421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bca02c-fadb-4d7e-86d3-719ef8fd68a1 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:41.254150 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1f33e-1e90-46ca-a44a-f5898e9b229b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:41.286912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d2612-e7b1-494b-86e2-723dfe9c99d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:41.287584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d2612-e7b1-494b-86e2-723dfe9c99d9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:41.287878 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d2612-e7b1-494b-86e2-723dfe9c99d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:41.288162 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d2612-e7b1-494b-86e2-723dfe9c99d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:41.289485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d2612-e7b1-494b-86e2-723dfe9c99d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:41.289741 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d2612-e7b1-494b-86e2-723dfe9c99d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:41.291124 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d2612-e7b1-494b-86e2-723dfe9c99d9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:41.299092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 40/81] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:20:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:41.299879 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d2612-e7b1-494b-86e2-723dfe9c99d9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:41.356478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 42/82] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:20:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:20:41.357088 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bca02c-fadb-4d7e-86d3-719ef8fd68a1 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:41.388738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:41.401083 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d2612-e7b1-494b-86e2-723dfe9c99d9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:41.417144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:41.435082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78da61f-b26e-40de-a00a-fcf2347d2060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:41.435746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78da61f-b26e-40de-a00a-fcf2347d2060 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:41.436048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78da61f-b26e-40de-a00a-fcf2347d2060 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:41.436336 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78da61f-b26e-40de-a00a-fcf2347d2060 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:41.457273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bca02c-fadb-4d7e-86d3-719ef8fd68a1 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:41.493838 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99acd43-48de-40a3-85af-42be85e39d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:41.494537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99acd43-48de-40a3-85af-42be85e39d2c None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:41.494832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99acd43-48de-40a3-85af-42be85e39d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:41.495123 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99acd43-48de-40a3-85af-42be85e39d2c None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:41.495834 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99acd43-48de-40a3-85af-42be85e39d2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:41.497326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99acd43-48de-40a3-85af-42be85e39d2c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:41.506573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78da61f-b26e-40de-a00a-fcf2347d2060 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:41.509038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78da61f-b26e-40de-a00a-fcf2347d2060 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:41.602480 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 41/83] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:20:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:20:41.603317 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78da61f-b26e-40de-a00a-fcf2347d2060 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:41.610254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 43/84] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:20:41 2022] POST /identity/v3/users => generated 289 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:20:41.611041 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99acd43-48de-40a3-85af-42be85e39d2c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:41.630111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:41.719347 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:41.730994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78da61f-b26e-40de-a00a-fcf2347d2060 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:41.732147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99acd43-48de-40a3-85af-42be85e39d2c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:41.771615 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc12e23-c47a-4415-b738-268bf52a35e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:41.771826 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6aa39-1599-4869-a6eb-f47567b18f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:41.772430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6aa39-1599-4869-a6eb-f47567b18f23 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:41.772553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc12e23-c47a-4415-b738-268bf52a35e5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:41.772725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6aa39-1599-4869-a6eb-f47567b18f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:41.772833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc12e23-c47a-4415-b738-268bf52a35e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:41.773002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6aa39-1599-4869-a6eb-f47567b18f23 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:41.773161 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc12e23-c47a-4415-b738-268bf52a35e5 None admin] PATH_INFO: `/v3/users/358e903ec0d14cd2809ef65ffb8035f5` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:41.774471 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6aa39-1599-4869-a6eb-f47567b18f23 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:41.774585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc12e23-c47a-4415-b738-268bf52a35e5 None admin] RBAC: Authorizing `identity:get_user(user_id=358e903ec0d14cd2809ef65ffb8035f5)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:41.774793 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6aa39-1599-4869-a6eb-f47567b18f23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:41.775967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc12e23-c47a-4415-b738-268bf52a35e5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:41.776179 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6aa39-1599-4869-a6eb-f47567b18f23 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:41.777711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 44/85] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:41 2022] GET /identity/v3/users/358e903ec0d14cd2809ef65ffb8035f5 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:41.780619 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc12e23-c47a-4415-b738-268bf52a35e5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:41.789296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 42/86] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:20:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:41.790452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6aa39-1599-4869-a6eb-f47567b18f23 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:41.891575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:41.902099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc12e23-c47a-4415-b738-268bf52a35e5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:41.921464 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6aa39-1599-4869-a6eb-f47567b18f23 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:41.949344 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174109f-9d36-409f-93f6-aa2224823f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:41.950240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174109f-9d36-409f-93f6-aa2224823f22 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:41.950530 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174109f-9d36-409f-93f6-aa2224823f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:41.950818 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174109f-9d36-409f-93f6-aa2224823f22 None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:41.952197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174109f-9d36-409f-93f6-aa2224823f22 None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:41.953694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174109f-9d36-409f-93f6-aa2224823f22 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:41.955422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 45/87] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:41 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:41.956299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174109f-9d36-409f-93f6-aa2224823f22 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:41.963209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37d260-c139-454c-9a79-5ee4a6267f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:41.964159 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37d260-c139-454c-9a79-5ee4a6267f50 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:41.964461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37d260-c139-454c-9a79-5ee4a6267f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:41.965004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37d260-c139-454c-9a79-5ee4a6267f50 None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:41.966676 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37d260-c139-454c-9a79-5ee4a6267f50 None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:41.967699 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37d260-c139-454c-9a79-5ee4a6267f50 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:41.970508 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 43/88] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:41 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:41.971179 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37d260-c139-454c-9a79-5ee4a6267f50 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:41.985887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:42.087486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174109f-9d36-409f-93f6-aa2224823f22 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:42.090949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37d260-c139-454c-9a79-5ee4a6267f50 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:42.129994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8f051-d98e-4ef7-a391-bfb658c3897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:42.131075 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8f051-d98e-4ef7-a391-bfb658c3897a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:42.131473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8f051-d98e-4ef7-a391-bfb658c3897a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:42.131863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8f051-d98e-4ef7-a391-bfb658c3897a None admin] PATH_INFO: `/v3/users/358e903ec0d14cd2809ef65ffb8035f5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:42.132059 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918692d-8eda-4dce-8207-1a86a31e541c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:42.133597 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8f051-d98e-4ef7-a391-bfb658c3897a None admin] RBAC: Authorizing `identity:get_user(user_id=358e903ec0d14cd2809ef65ffb8035f5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:42.133866 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918692d-8eda-4dce-8207-1a86a31e541c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:42.134377 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918692d-8eda-4dce-8207-1a86a31e541c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:42.134807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918692d-8eda-4dce-8207-1a86a31e541c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:42.135595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8f051-d98e-4ef7-a391-bfb658c3897a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:42.139146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 44/89] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:41 2022] GET /identity/v3/users/358e903ec0d14cd2809ef65ffb8035f5 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:42.139146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8f051-d98e-4ef7-a391-bfb658c3897a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:42.161361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:42.232806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918692d-8eda-4dce-8207-1a86a31e541c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:42.235182 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918692d-8eda-4dce-8207-1a86a31e541c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:42.258257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8f051-d98e-4ef7-a391-bfb658c3897a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:42.267725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:42.298618 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303be08-3819-4abb-91e3-007ee36048b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:42.299373 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303be08-3819-4abb-91e3-007ee36048b0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:42.299676 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303be08-3819-4abb-91e3-007ee36048b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:42.299962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303be08-3819-4abb-91e3-007ee36048b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:42.300946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303be08-3819-4abb-91e3-007ee36048b0 None admin] RBAC: Adding query filter params (role.id=9330473f62ec4a82a82d1c4beb6160d7, scope.project.id=936684d8d665414698e461a87200ef9b, user.id=358e903ec0d14cd2809ef65ffb8035f5) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:42.301279 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303be08-3819-4abb-91e3-007ee36048b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:42.302755 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303be08-3819-4abb-91e3-007ee36048b0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:42.317086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 45/90] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:20:42 2022] GET /identity/v3/role_assignments?user.id=358e903ec0d14cd2809ef65ffb8035f5&scope.project.id=936684d8d665414698e461a87200ef9b&role.id=9330473f62ec4a82a82d1c4beb6160d7 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:42.317086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303be08-3819-4abb-91e3-007ee36048b0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:42.407801 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 46/91] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:20:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:20:42.409433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918692d-8eda-4dce-8207-1a86a31e541c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:42.450413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:42.474553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303be08-3819-4abb-91e3-007ee36048b0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:42.521100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bea08b-c70f-4dea-b8bf-311f5bec6dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:42.522157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bea08b-c70f-4dea-b8bf-311f5bec6dff None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:42.523949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bea08b-c70f-4dea-b8bf-311f5bec6dff None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:42.524628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bea08b-c70f-4dea-b8bf-311f5bec6dff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:42.527068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bea08b-c70f-4dea-b8bf-311f5bec6dff None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=936684d8d665414698e461a87200ef9b, user.id=2781ce56481d4d179e62cc19ea41a8b0) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:42.527661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bea08b-c70f-4dea-b8bf-311f5bec6dff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:42.530249 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bea08b-c70f-4dea-b8bf-311f5bec6dff None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:42.543966 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 46/92] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:20:42 2022] GET /identity/v3/role_assignments?user.id=2781ce56481d4d179e62cc19ea41a8b0&scope.project.id=936684d8d665414698e461a87200ef9b&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:42.546758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bea08b-c70f-4dea-b8bf-311f5bec6dff None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:42.618419 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918692d-8eda-4dce-8207-1a86a31e541c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:42.657450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:42.680526 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd3521-0dae-46cb-a049-3cc69c2dcaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:42.681239 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd3521-0dae-46cb-a049-3cc69c2dcaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:42.681538 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd3521-0dae-46cb-a049-3cc69c2dcaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:42.681825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd3521-0dae-46cb-a049-3cc69c2dcaf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:42.732441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bea08b-c70f-4dea-b8bf-311f5bec6dff None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:42.800741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd3521-0dae-46cb-a049-3cc69c2dcaf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:42.801857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b03dcc-98bc-435a-83fa-fd67d2f6180e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:42.802301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd3521-0dae-46cb-a049-3cc69c2dcaf7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:42.802790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b03dcc-98bc-435a-83fa-fd67d2f6180e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:42.803081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b03dcc-98bc-435a-83fa-fd67d2f6180e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:42.803360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b03dcc-98bc-435a-83fa-fd67d2f6180e None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:42.806079 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b03dcc-98bc-435a-83fa-fd67d2f6180e None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:42.807556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b03dcc-98bc-435a-83fa-fd67d2f6180e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:42.809380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 47/93] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:42 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:42.815298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b03dcc-98bc-435a-83fa-fd67d2f6180e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:42.841471 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:42.994447 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b03dcc-98bc-435a-83fa-fd67d2f6180e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:43.008438 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 47/94] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:20:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:20:43.014841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd3521-0dae-46cb-a049-3cc69c2dcaf7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:43.041561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b070c-3801-48f7-a479-fcb3b6590a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:43.042238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b070c-3801-48f7-a479-fcb3b6590a6b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:43.042531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b070c-3801-48f7-a479-fcb3b6590a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:43.042817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b070c-3801-48f7-a479-fcb3b6590a6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:43.043812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b070c-3801-48f7-a479-fcb3b6590a6b None admin] RBAC: Adding query filter params (role.id=1bac0428b2544e2cb16fad539f080c14, scope.project.id=936684d8d665414698e461a87200ef9b, user.id=358e903ec0d14cd2809ef65ffb8035f5) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:43.044002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b070c-3801-48f7-a479-fcb3b6590a6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:43.049085 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b070c-3801-48f7-a479-fcb3b6590a6b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:43.058322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:43.061021 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 48/95] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:20:42 2022] GET /identity/v3/role_assignments?user.id=358e903ec0d14cd2809ef65ffb8035f5&scope.project.id=936684d8d665414698e461a87200ef9b&role.id=1bac0428b2544e2cb16fad539f080c14 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:43.179756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd3521-0dae-46cb-a049-3cc69c2dcaf7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:43.232177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d37b0-8be9-4def-a9bb-1d1b68774b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:43.233380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d37b0-8be9-4def-a9bb-1d1b68774b25 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:43.233851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d37b0-8be9-4def-a9bb-1d1b68774b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:43.234373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d37b0-8be9-4def-a9bb-1d1b68774b25 None admin] PATH_INFO: `/v3/projects/5575c0a712014ebc84e5feaed1ff36df` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:43.241983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d37b0-8be9-4def-a9bb-1d1b68774b25 None admin] RBAC: Authorizing `identity:get_project(project_id=5575c0a712014ebc84e5feaed1ff36df)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:43.243836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d37b0-8be9-4def-a9bb-1d1b68774b25 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:43.246372 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 48/96] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:43 2022] GET /identity/v3/projects/5575c0a712014ebc84e5feaed1ff36df => generated 312 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:43.251934 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b070c-3801-48f7-a479-fcb3b6590a6b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:43.361454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:43.389809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b070c-3801-48f7-a479-fcb3b6590a6b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:43.439428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fdddb-3300-455d-947b-75e0bb777a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:43.440121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fdddb-3300-455d-947b-75e0bb777a09 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:43.440414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fdddb-3300-455d-947b-75e0bb777a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:43.440700 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fdddb-3300-455d-947b-75e0bb777a09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:43.441583 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fdddb-3300-455d-947b-75e0bb777a09 None admin] RBAC: Adding query filter params (role.id=9330473f62ec4a82a82d1c4beb6160d7, scope.project.id=5575c0a712014ebc84e5feaed1ff36df, user.id=358e903ec0d14cd2809ef65ffb8035f5) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:43.441902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fdddb-3300-455d-947b-75e0bb777a09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:43.443366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fdddb-3300-455d-947b-75e0bb777a09 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:43.450517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 49/97] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:20:43 2022] GET /identity/v3/role_assignments?user.id=358e903ec0d14cd2809ef65ffb8035f5&scope.project.id=5575c0a712014ebc84e5feaed1ff36df&role.id=9330473f62ec4a82a82d1c4beb6160d7 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:44.348627 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda5e90-ac5a-4f0e-83a6-71671f53eef6 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:44.349701 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda5e90-ac5a-4f0e-83a6-71671f53eef6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:44.350199 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda5e90-ac5a-4f0e-83a6-71671f53eef6 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:44.352430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 49/98] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:44.357909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a4408-8b82-4f8b-9b4b-5b003a5007d0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:44.358370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a4408-8b82-4f8b-9b4b-5b003a5007d0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:44.358479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a4408-8b82-4f8b-9b4b-5b003a5007d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:44.379630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-bd8a4408-8b82-4f8b-9b4b-5b003a5007d0 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:44.381110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bd8a4408-8b82-4f8b-9b4b-5b003a5007d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:44.409663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af00e97-3711-4221-a58c-59859c0b8c26 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:44.410326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af00e97-3711-4221-a58c-59859c0b8c26 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:44.410850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af00e97-3711-4221-a58c-59859c0b8c26 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:44.412487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 50/99] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:44.417926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e4be3-b2c6-4f37-b3ee-fb343e8c6e62 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:44.418476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e4be3-b2c6-4f37-b3ee-fb343e8c6e62 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:44.418982 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e4be3-b2c6-4f37-b3ee-fb343e8c6e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:44.438678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-984e4be3-b2c6-4f37-b3ee-fb343e8c6e62 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:44.440907 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-984e4be3-b2c6-4f37-b3ee-fb343e8c6e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:44.490102 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 50/100] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:44.514390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a4408-8b82-4f8b-9b4b-5b003a5007d0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:44.550797 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 51/101] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:20:44.597017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:44.612659 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e4be3-b2c6-4f37-b3ee-fb343e8c6e62 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:44.658612 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:44.677082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a4408-8b82-4f8b-9b4b-5b003a5007d0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:44.727523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946afdf-ef55-4136-8953-8e59a07f871c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:44.728602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946afdf-ef55-4136-8953-8e59a07f871c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:44.729043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946afdf-ef55-4136-8953-8e59a07f871c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:44.729492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946afdf-ef55-4136-8953-8e59a07f871c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:44.740476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946afdf-ef55-4136-8953-8e59a07f871c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b946afdf-ef55-4136-8953-8e59a07f871c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:44.749173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:44.750334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:44.751253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:44.751253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:44.751253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:44.751253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:44.751253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:44.751253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:44.751253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:44.751253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:44.751253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:44.751253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:44.751253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:20:44.751253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:20:44.753399 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 51/102] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:20:44.762059 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946afdf-ef55-4136-8953-8e59a07f871c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:44.839115 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e4be3-b2c6-4f37-b3ee-fb343e8c6e62 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:44.885186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946afdf-ef55-4136-8953-8e59a07f871c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:44.915071 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:44.921239 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75c00e-1c6b-4c4f-8db6-813d1bbc0458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:44.922123 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75c00e-1c6b-4c4f-8db6-813d1bbc0458 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:44.922597 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75c00e-1c6b-4c4f-8db6-813d1bbc0458 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:44.923061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75c00e-1c6b-4c4f-8db6-813d1bbc0458 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:44.924015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75c00e-1c6b-4c4f-8db6-813d1bbc0458 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:44.924625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75c00e-1c6b-4c4f-8db6-813d1bbc0458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:44.926281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75c00e-1c6b-4c4f-8db6-813d1bbc0458 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:44.936467 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 52/103] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:20:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:44.942312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75c00e-1c6b-4c4f-8db6-813d1bbc0458 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:44.942504 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56ca35-c97a-4318-9fd9-7da1af672030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:44.943303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56ca35-c97a-4318-9fd9-7da1af672030 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:44.943619 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56ca35-c97a-4318-9fd9-7da1af672030 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:44.943882 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56ca35-c97a-4318-9fd9-7da1af672030 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:44.949791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56ca35-c97a-4318-9fd9-7da1af672030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:44.951294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56ca35-c97a-4318-9fd9-7da1af672030 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:44.952963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 52/104] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:20:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:44.974775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56ca35-c97a-4318-9fd9-7da1af672030 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:45.051854 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75c00e-1c6b-4c4f-8db6-813d1bbc0458 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:45.088642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb4dbb-561c-49e2-ba91-bdbc50e2a2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:45.089946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb4dbb-561c-49e2-ba91-bdbc50e2a2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:45.090418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb4dbb-561c-49e2-ba91-bdbc50e2a2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:45.090882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb4dbb-561c-49e2-ba91-bdbc50e2a2f9 None admin] PATH_INFO: `/v3/users/38275baf180342959a6ee609014b2f1f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:45.106096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb4dbb-561c-49e2-ba91-bdbc50e2a2f9 None admin] RBAC: Authorizing `identity:get_user(user_id=38275baf180342959a6ee609014b2f1f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:45.107926 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb4dbb-561c-49e2-ba91-bdbc50e2a2f9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:45.110190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 53/105] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:44 2022] GET /identity/v3/users/38275baf180342959a6ee609014b2f1f => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:45.116305 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb4dbb-561c-49e2-ba91-bdbc50e2a2f9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:45.170454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:45.207141 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:45.225274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56ca35-c97a-4318-9fd9-7da1af672030 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:45.227456 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb4dbb-561c-49e2-ba91-bdbc50e2a2f9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:45.267190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4a43a-ebff-469f-a224-8024296e500c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:45.268258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4a43a-ebff-469f-a224-8024296e500c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:45.268867 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4a43a-ebff-469f-a224-8024296e500c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:45.269425 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4a43a-ebff-469f-a224-8024296e500c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:45.287399 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81dd38-0d01-4897-b631-6982ae455e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:45.288176 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81dd38-0d01-4897-b631-6982ae455e8e None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:45.289275 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81dd38-0d01-4897-b631-6982ae455e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:45.289904 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81dd38-0d01-4897-b631-6982ae455e8e None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:45.290718 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81dd38-0d01-4897-b631-6982ae455e8e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:45.292470 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81dd38-0d01-4897-b631-6982ae455e8e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:45.347669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4a43a-ebff-469f-a224-8024296e500c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:45.349663 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4a43a-ebff-469f-a224-8024296e500c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:45.406324 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:45.453418 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 53/106] 10.222.0.64 () {60 vars in 1141 bytes} [Mon May 23 15:20:44 2022] POST /identity/v3/projects => generated 302 bytes in 479 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:20:45.463831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 54/107] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:20:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:20:45.470051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81dd38-0d01-4897-b631-6982ae455e8e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:45.622569 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81dd38-0d01-4897-b631-6982ae455e8e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:45.717170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e2abb-ca4d-44d0-8150-4bd4db67f912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:45.718052 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e2abb-ca4d-44d0-8150-4bd4db67f912 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:45.718348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e2abb-ca4d-44d0-8150-4bd4db67f912 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:45.718640 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e2abb-ca4d-44d0-8150-4bd4db67f912 None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:45.719905 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e2abb-ca4d-44d0-8150-4bd4db67f912 None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:45.721440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e2abb-ca4d-44d0-8150-4bd4db67f912 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:45.723346 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 54/108] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:45 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:45.730042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4a43a-ebff-469f-a224-8024296e500c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:45.841328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4a43a-ebff-469f-a224-8024296e500c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:45.887366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d85fe7-12a6-4fcc-a0f8-d8cf73b37297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:45.888062 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d85fe7-12a6-4fcc-a0f8-d8cf73b37297 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:45.888435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d85fe7-12a6-4fcc-a0f8-d8cf73b37297 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:45.888730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d85fe7-12a6-4fcc-a0f8-d8cf73b37297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:45.889634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d85fe7-12a6-4fcc-a0f8-d8cf73b37297 None admin] RBAC: Adding query filter params (role.id=1026857a0bd64410a463448b53532814, scope.project.id=936684d8d665414698e461a87200ef9b, user.id=38275baf180342959a6ee609014b2f1f) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:45.889959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d85fe7-12a6-4fcc-a0f8-d8cf73b37297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:45.891466 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d85fe7-12a6-4fcc-a0f8-d8cf73b37297 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:45.902439 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 55/109] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:20:45 2022] GET /identity/v3/role_assignments?user.id=38275baf180342959a6ee609014b2f1f&scope.project.id=936684d8d665414698e461a87200ef9b&role.id=1026857a0bd64410a463448b53532814 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:46.025891 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb84af-e54f-4715-b3a1-7e87e79469bd None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:46.026525 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb84af-e54f-4715-b3a1-7e87e79469bd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:46.027027 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb84af-e54f-4715-b3a1-7e87e79469bd None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:46.028729 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 55/110] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:46.034704 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ac182-3c60-4837-858e-77914261abab None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:46.035317 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ac182-3c60-4837-858e-77914261abab None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:46.035808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ac182-3c60-4837-858e-77914261abab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:46.057949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-177ac182-3c60-4837-858e-77914261abab None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:46.059600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-177ac182-3c60-4837-858e-77914261abab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:46.068443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:46.150160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 56/111] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:46.184178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb84af-e54f-4715-b3a1-7e87e79469bd None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:46.198888 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:46.264180 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c09203-1604-40e2-8402-b2759a3b5c9a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:46.264844 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c09203-1604-40e2-8402-b2759a3b5c9a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:46.265358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c09203-1604-40e2-8402-b2759a3b5c9a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:46.267029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 57/112] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:20:46.276277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6a051-d109-4645-8f21-1e386e5bdf57 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:46.276864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6a051-d109-4645-8f21-1e386e5bdf57 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:46.277372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6a051-d109-4645-8f21-1e386e5bdf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:46.301727 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8da6a051-d109-4645-8f21-1e386e5bdf57 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:46.303377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8da6a051-d109-4645-8f21-1e386e5bdf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:46.342952 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb84af-e54f-4715-b3a1-7e87e79469bd None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:46.394516 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 58/113] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:20:46.399792 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47421e92-f4d4-401b-8c52-af83a7df4f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:46.400849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47421e92-f4d4-401b-8c52-af83a7df4f6a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:46.401384 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47421e92-f4d4-401b-8c52-af83a7df4f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:46.401903 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47421e92-f4d4-401b-8c52-af83a7df4f6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:46.419120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47421e92-f4d4-401b-8c52-af83a7df4f6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-47421e92-f4d4-401b-8c52-af83a7df4f6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:46.427566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:46.428449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:46.430110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:46.430110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:46.430110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:46.430110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:46.430110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:46.430110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:46.430110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:46.430110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:46.430110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:46.430110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:46.430110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:20:46.430110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:20:46.432241 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6a051-d109-4645-8f21-1e386e5bdf57 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:46.452225 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 56/114] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:46.458049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47421e92-f4d4-401b-8c52-af83a7df4f6a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:46.597433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:46.633974 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:46.659414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6a051-d109-4645-8f21-1e386e5bdf57 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:46.691580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47421e92-f4d4-401b-8c52-af83a7df4f6a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:46.712078 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99033f91-e852-4cc9-96f0-eed1ba15ecd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:46.713026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99033f91-e852-4cc9-96f0-eed1ba15ecd7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:46.713260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99033f91-e852-4cc9-96f0-eed1ba15ecd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:46.713508 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99033f91-e852-4cc9-96f0-eed1ba15ecd7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:46.730479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99033f91-e852-4cc9-96f0-eed1ba15ecd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:46.743048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d75be-3160-4b67-b82d-6b15982e4e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:46.743993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d75be-3160-4b67-b82d-6b15982e4e99 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:46.744466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d75be-3160-4b67-b82d-6b15982e4e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:46.744951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d75be-3160-4b67-b82d-6b15982e4e99 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:46.745981 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d75be-3160-4b67-b82d-6b15982e4e99 None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:46.746521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d75be-3160-4b67-b82d-6b15982e4e99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-99033f91-e852-4cc9-96f0-eed1ba15ecd7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:46.747258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:46.748126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:46.748913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:46.748913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:46.748913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:46.748913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:46.748913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:46.748913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:46.748913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:46.748913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:46.748913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:46.748913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:46.748913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:46.748913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:20:46.748913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 59/115] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:20:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:46.751124 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d75be-3160-4b67-b82d-6b15982e4e99 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:46.758657 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99033f91-e852-4cc9-96f0-eed1ba15ecd7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:46.763063 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 57/116] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:20:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:46.768854 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d75be-3160-4b67-b82d-6b15982e4e99 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:46.917169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99033f91-e852-4cc9-96f0-eed1ba15ecd7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:46.931384 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d75be-3160-4b67-b82d-6b15982e4e99 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:46.942979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:46.949513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0d79e-0c32-48fd-971e-919c385bad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:46.950207 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0d79e-0c32-48fd-971e-919c385bad69 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:46.950507 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0d79e-0c32-48fd-971e-919c385bad69 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:46.950801 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0d79e-0c32-48fd-971e-919c385bad69 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:46.951577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0d79e-0c32-48fd-971e-919c385bad69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:46.951899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0d79e-0c32-48fd-971e-919c385bad69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:46.953443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0d79e-0c32-48fd-971e-919c385bad69 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:46.963185 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 60/117] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:46.964417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efe08c-d880-4d6f-bd2c-26684cbee8f9 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:46.964709 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efe08c-d880-4d6f-bd2c-26684cbee8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:46.964999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efe08c-d880-4d6f-bd2c-26684cbee8f9 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:46.966274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 61/118] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:46.967360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:46.974101 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3fd91-0459-412b-9980-1c700a07a637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:46.975645 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3fd91-0459-412b-9980-1c700a07a637 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:46.976366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3fd91-0459-412b-9980-1c700a07a637 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:46.976883 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3fd91-0459-412b-9980-1c700a07a637 None admin] PATH_INFO: `/v3/users/358e903ec0d14cd2809ef65ffb8035f5` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:46.978213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efe08c-d880-4d6f-bd2c-26684cbee8f9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:46.978856 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3fd91-0459-412b-9980-1c700a07a637 None admin] RBAC: Authorizing `identity:get_user(user_id=358e903ec0d14cd2809ef65ffb8035f5)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:46.981353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3fd91-0459-412b-9980-1c700a07a637 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:46.983365 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 58/119] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:46 2022] GET /identity/v3/users/358e903ec0d14cd2809ef65ffb8035f5 => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:46.984826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7d0ee-26ae-49c5-896d-6d00a08161c8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:46.985358 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7d0ee-26ae-49c5-896d-6d00a08161c8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:46.985847 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7d0ee-26ae-49c5-896d-6d00a08161c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:47.003990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-27e7d0ee-26ae-49c5-896d-6d00a08161c8 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:47.005567 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-27e7d0ee-26ae-49c5-896d-6d00a08161c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:47.080763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 59/120] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:47.082443 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca9026-5c7e-415b-a932-44dbf0b09575 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:47.082961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca9026-5c7e-415b-a932-44dbf0b09575 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:47.083407 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca9026-5c7e-415b-a932-44dbf0b09575 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:47.084821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 60/121] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:47.086631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca9026-5c7e-415b-a932-44dbf0b09575 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:47.126345 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efe08c-d880-4d6f-bd2c-26684cbee8f9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:47.165015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119a120-4274-4eb5-b11c-ac348cd3a7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:47.165941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119a120-4274-4eb5-b11c-ac348cd3a7da None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:47.166239 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119a120-4274-4eb5-b11c-ac348cd3a7da None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:47.166528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119a120-4274-4eb5-b11c-ac348cd3a7da None admin] PATH_INFO: `/v3/users/2781ce56481d4d179e62cc19ea41a8b0` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:47.167813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119a120-4274-4eb5-b11c-ac348cd3a7da None admin] RBAC: Authorizing `identity:get_user(user_id=2781ce56481d4d179e62cc19ea41a8b0)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:47.169332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119a120-4274-4eb5-b11c-ac348cd3a7da None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:47.171272 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 62/122] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:46 2022] GET /identity/v3/users/2781ce56481d4d179e62cc19ea41a8b0 => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:47.172315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3ebc8-c6e9-4b41-98f8-0739e4b8827e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:47.172616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3ebc8-c6e9-4b41-98f8-0739e4b8827e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:47.172903 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3ebc8-c6e9-4b41-98f8-0739e4b8827e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:47.190748 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b0d3ebc8-c6e9-4b41-98f8-0739e4b8827e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:47.191947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b0d3ebc8-c6e9-4b41-98f8-0739e4b8827e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:47.200838 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca9026-5c7e-415b-a932-44dbf0b09575 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:47.232168 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:47.238997 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8e93f-ea6e-4e6d-bbb9-1403705285f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:47.240352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8e93f-ea6e-4e6d-bbb9-1403705285f0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:47.240877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8e93f-ea6e-4e6d-bbb9-1403705285f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:47.241486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8e93f-ea6e-4e6d-bbb9-1403705285f0 None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:47.243416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8e93f-ea6e-4e6d-bbb9-1403705285f0 None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:47.245637 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8e93f-ea6e-4e6d-bbb9-1403705285f0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:47.247806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 61/123] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:47 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:47.249181 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8e93f-ea6e-4e6d-bbb9-1403705285f0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:47.254048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:47.266294 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 63/124] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:47.267487 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3ebc8-c6e9-4b41-98f8-0739e4b8827e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:47.360407 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8e93f-ea6e-4e6d-bbb9-1403705285f0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:47.372301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3ebc8-c6e9-4b41-98f8-0739e4b8827e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:47.407931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5f5e1-72a6-4aa7-9de1-6264fdad733c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:47.409103 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5f5e1-72a6-4aa7-9de1-6264fdad733c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:47.409512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5f5e1-72a6-4aa7-9de1-6264fdad733c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:47.409897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5f5e1-72a6-4aa7-9de1-6264fdad733c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:47.410477 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddc037-4d9d-4879-8dd0-c82b5544696b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:47.411510 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddc037-4d9d-4879-8dd0-c82b5544696b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:47.411938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddc037-4d9d-4879-8dd0-c82b5544696b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:47.412407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddc037-4d9d-4879-8dd0-c82b5544696b None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:47.418221 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddc037-4d9d-4879-8dd0-c82b5544696b None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:47.418221 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddc037-4d9d-4879-8dd0-c82b5544696b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:47.418221 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 64/125] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:47 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:47.418221 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddc037-4d9d-4879-8dd0-c82b5544696b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:47.428083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5f5e1-72a6-4aa7-9de1-6264fdad733c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-07b5f5e1-72a6-4aa7-9de1-6264fdad733c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:47.436653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:47.437587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:47.438800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:47.438800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:47.438800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:47.438800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:47.438800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:47.438800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:47.438800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:47.438800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:47.438800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:47.438800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:47.438800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:20:47.438800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:20:47.438800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 62/126] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:47.439446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5f5e1-72a6-4aa7-9de1-6264fdad733c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:47.494691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:47.516672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:47.557318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddc037-4d9d-4879-8dd0-c82b5544696b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:47.572717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5f5e1-72a6-4aa7-9de1-6264fdad733c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:47.601871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828ac61-9392-4361-afed-9348671eb124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:47.603048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828ac61-9392-4361-afed-9348671eb124 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:47.603735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828ac61-9392-4361-afed-9348671eb124 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:47.604128 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828ac61-9392-4361-afed-9348671eb124 None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b/users/358e903ec0d14cd2809ef65ffb8035f5/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:47.608252 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828ac61-9392-4361-afed-9348671eb124 None admin] RBAC: Authorizing `identity:create_grant(project_id=936684d8d665414698e461a87200ef9b, user_id=358e903ec0d14cd2809ef65ffb8035f5, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:47.612254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828ac61-9392-4361-afed-9348671eb124 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:47.616983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c53b8-1733-45a5-9f3c-6192a7289255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:47.618112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c53b8-1733-45a5-9f3c-6192a7289255 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:47.619061 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c53b8-1733-45a5-9f3c-6192a7289255 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:47.619354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c53b8-1733-45a5-9f3c-6192a7289255 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:47.631618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c53b8-1733-45a5-9f3c-6192a7289255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-df3c53b8-1733-45a5-9f3c-6192a7289255 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:47.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:47.643108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:47.643108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:47.643108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:47.643108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:47.643108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:47.643108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:47.643108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:47.643108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:47.643108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:47.643108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:47.643108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:20:47.643108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:20:47.643108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 63/127] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:20:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:47.644808 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c53b8-1733-45a5-9f3c-6192a7289255 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:47.718668 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 65/128] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:20:47 2022] PUT /identity/v3/projects/936684d8d665414698e461a87200ef9b/users/358e903ec0d14cd2809ef65ffb8035f5/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:20:47.719990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828ac61-9392-4361-afed-9348671eb124 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:47.774296 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c53b8-1733-45a5-9f3c-6192a7289255 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:47.798289 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:47.823474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd534db-3fab-4599-a334-4760bc842955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:47.824873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd534db-3fab-4599-a334-4760bc842955 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:47.825357 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd534db-3fab-4599-a334-4760bc842955 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:47.826529 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd534db-3fab-4599-a334-4760bc842955 None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b/users/2781ce56481d4d179e62cc19ea41a8b0/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:47.832229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd534db-3fab-4599-a334-4760bc842955 None admin] RBAC: Authorizing `identity:create_grant(project_id=936684d8d665414698e461a87200ef9b, user_id=2781ce56481d4d179e62cc19ea41a8b0, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:47.836209 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd534db-3fab-4599-a334-4760bc842955 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:47.915843 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 64/129] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:20:47 2022] PUT /identity/v3/projects/936684d8d665414698e461a87200ef9b/users/2781ce56481d4d179e62cc19ea41a8b0/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:20:47.916942 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd534db-3fab-4599-a334-4760bc842955 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:47.926391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828ac61-9392-4361-afed-9348671eb124 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:47.978231 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:48.013882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae6024-24c2-4d26-8766-5676d0c94e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:48.014942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae6024-24c2-4d26-8766-5676d0c94e74 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:48.015351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae6024-24c2-4d26-8766-5676d0c94e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:48.016677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae6024-24c2-4d26-8766-5676d0c94e74 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:48.020447 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae6024-24c2-4d26-8766-5676d0c94e74 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:48.020834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae6024-24c2-4d26-8766-5676d0c94e74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:48.022575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae6024-24c2-4d26-8766-5676d0c94e74 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:48.043087 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 66/130] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:20:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:48.051562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae6024-24c2-4d26-8766-5676d0c94e74 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:48.119991 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd534db-3fab-4599-a334-4760bc842955 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:48.173432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602c765-d0eb-4029-8127-7fc23c6d6969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:48.174379 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602c765-d0eb-4029-8127-7fc23c6d6969 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:48.174681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602c765-d0eb-4029-8127-7fc23c6d6969 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:48.175037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602c765-d0eb-4029-8127-7fc23c6d6969 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:48.176267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602c765-d0eb-4029-8127-7fc23c6d6969 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:48.176596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602c765-d0eb-4029-8127-7fc23c6d6969 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:48.179309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602c765-d0eb-4029-8127-7fc23c6d6969 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:48.185808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae6024-24c2-4d26-8766-5676d0c94e74 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:48.191156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 65/131] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:20:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:48.204219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602c765-d0eb-4029-8127-7fc23c6d6969 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:48.269466 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798717af-f03d-428a-9933-b212581f853e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:48.269991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798717af-f03d-428a-9933-b212581f853e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:48.270282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798717af-f03d-428a-9933-b212581f853e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:48.270560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798717af-f03d-428a-9933-b212581f853e None admin] PATH_INFO: `/v3/users/358e903ec0d14cd2809ef65ffb8035f5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:48.272075 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798717af-f03d-428a-9933-b212581f853e None admin] RBAC: Authorizing `identity:get_user(user_id=358e903ec0d14cd2809ef65ffb8035f5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:48.280148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798717af-f03d-428a-9933-b212581f853e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:48.282556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 67/132] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:48 2022] GET /identity/v3/users/358e903ec0d14cd2809ef65ffb8035f5 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:48.291472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798717af-f03d-428a-9933-b212581f853e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:48.322554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:48.439572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:48.458794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602c765-d0eb-4029-8127-7fc23c6d6969 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:48.508920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f318f48-4715-4466-893b-b93539a3b7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:48.509870 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f318f48-4715-4466-893b-b93539a3b7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:48.510166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f318f48-4715-4466-893b-b93539a3b7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:48.510453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f318f48-4715-4466-893b-b93539a3b7c5 None admin] PATH_INFO: `/v3/users/358e903ec0d14cd2809ef65ffb8035f5` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:48.510869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798717af-f03d-428a-9933-b212581f853e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:48.511760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f318f48-4715-4466-893b-b93539a3b7c5 None admin] RBAC: Authorizing `identity:get_user(user_id=358e903ec0d14cd2809ef65ffb8035f5)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:48.513503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f318f48-4715-4466-893b-b93539a3b7c5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:48.516166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 66/133] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:48 2022] GET /identity/v3/users/358e903ec0d14cd2809ef65ffb8035f5 => generated 275 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:48.521789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f318f48-4715-4466-893b-b93539a3b7c5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:48.554565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e943de-2eff-4074-96f9-737356eb0658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:48.555443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e943de-2eff-4074-96f9-737356eb0658 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:48.555737 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e943de-2eff-4074-96f9-737356eb0658 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:48.556017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e943de-2eff-4074-96f9-737356eb0658 None admin] PATH_INFO: `/v3/projects/5575c0a712014ebc84e5feaed1ff36df` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:48.557289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e943de-2eff-4074-96f9-737356eb0658 None admin] RBAC: Authorizing `identity:get_project(project_id=5575c0a712014ebc84e5feaed1ff36df)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:48.558762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e943de-2eff-4074-96f9-737356eb0658 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:48.560822 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 68/134] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:48 2022] GET /identity/v3/projects/5575c0a712014ebc84e5feaed1ff36df => generated 312 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:48.567646 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e943de-2eff-4074-96f9-737356eb0658 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:48.677425 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e943de-2eff-4074-96f9-737356eb0658 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:48.697457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:48.738005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f318f48-4715-4466-893b-b93539a3b7c5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:48.757435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:48.765852 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950c9e3-09f0-4e68-b1fb-b2b1926cef7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:48.766784 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950c9e3-09f0-4e68-b1fb-b2b1926cef7a None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:48.767073 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950c9e3-09f0-4e68-b1fb-b2b1926cef7a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:48.767390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950c9e3-09f0-4e68-b1fb-b2b1926cef7a None admin] PATH_INFO: `/v3/projects/5575c0a712014ebc84e5feaed1ff36df/users/358e903ec0d14cd2809ef65ffb8035f5/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:48.774955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1950c9e3-09f0-4e68-b1fb-b2b1926cef7a None admin] RBAC: Authorizing `identity:create_grant(project_id=5575c0a712014ebc84e5feaed1ff36df, user_id=358e903ec0d14cd2809ef65ffb8035f5, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:48.776431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1950c9e3-09f0-4e68-b1fb-b2b1926cef7a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:48.790243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a5707-4ad6-4ffb-be42-0deee781e09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:48.791144 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a5707-4ad6-4ffb-be42-0deee781e09e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:48.791433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a5707-4ad6-4ffb-be42-0deee781e09e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:48.791716 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a5707-4ad6-4ffb-be42-0deee781e09e None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:48.792978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a5707-4ad6-4ffb-be42-0deee781e09e None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:48.794540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a5707-4ad6-4ffb-be42-0deee781e09e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:48.796622 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 67/135] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:48 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:48.797789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbd6c9-962d-47a1-9a82-3b351b6eb8f9 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:48.798079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbd6c9-962d-47a1-9a82-3b351b6eb8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:48.798360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbd6c9-962d-47a1-9a82-3b351b6eb8f9 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:48.799758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 68/136] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:48.804925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbd6c9-962d-47a1-9a82-3b351b6eb8f9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:48.830095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 69/137] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:20:48 2022] PUT /identity/v3/projects/5575c0a712014ebc84e5feaed1ff36df/users/358e903ec0d14cd2809ef65ffb8035f5/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:20:48.831615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec197c-3016-4f55-b2c2-aa2b0bc86910 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:48.831916 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec197c-3016-4f55-b2c2-aa2b0bc86910 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:48.832249 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec197c-3016-4f55-b2c2-aa2b0bc86910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:48.869593 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4fec197c-3016-4f55-b2c2-aa2b0bc86910 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:48.871660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4fec197c-3016-4f55-b2c2-aa2b0bc86910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:48.969742 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbd6c9-962d-47a1-9a82-3b351b6eb8f9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:48.999434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 70/138] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:49.018385 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a696a4-eaca-41ca-9b65-82c0615c97df None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:49.018978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a696a4-eaca-41ca-9b65-82c0615c97df None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:49.019414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a696a4-eaca-41ca-9b65-82c0615c97df None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:49.020504 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e0d10-5edb-4c9c-9981-9a9206507e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:49.021092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 71/139] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:49.021440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e0d10-5edb-4c9c-9981-9a9206507e62 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:49.021734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e0d10-5edb-4c9c-9981-9a9206507e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:49.022016 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e0d10-5edb-4c9c-9981-9a9206507e62 None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b/users/358e903ec0d14cd2809ef65ffb8035f5/roles/1bac0428b2544e2cb16fad539f080c14` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:49.027228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e0d10-5edb-4c9c-9981-9a9206507e62 None admin] RBAC: Authorizing `identity:create_grant(project_id=936684d8d665414698e461a87200ef9b, user_id=358e903ec0d14cd2809ef65ffb8035f5, role_id=1bac0428b2544e2cb16fad539f080c14)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:49.028139 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc50aac-0ae0-410f-92cc-4740c79437d2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:49.028420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc50aac-0ae0-410f-92cc-4740c79437d2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:49.028652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e0d10-5edb-4c9c-9981-9a9206507e62 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:49.028743 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc50aac-0ae0-410f-92cc-4740c79437d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:49.053387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8dc50aac-0ae0-410f-92cc-4740c79437d2 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:49.054937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8dc50aac-0ae0-410f-92cc-4740c79437d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:49.111683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 69/140] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:20:48 2022] PUT /identity/v3/projects/936684d8d665414698e461a87200ef9b/users/358e903ec0d14cd2809ef65ffb8035f5/roles/1bac0428b2544e2cb16fad539f080c14 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:20:49.112558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e0d10-5edb-4c9c-9981-9a9206507e62 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:49.142559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 72/141] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:20:49.171552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:49.175341 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc50aac-0ae0-410f-92cc-4740c79437d2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:49.259659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:49.333322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e0d10-5edb-4c9c-9981-9a9206507e62 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:49.369156 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc50aac-0ae0-410f-92cc-4740c79437d2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:49.383022 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5898814-d1b3-45a1-988b-4235504da6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:49.383881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5898814-d1b3-45a1-988b-4235504da6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:49.384238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5898814-d1b3-45a1-988b-4235504da6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:49.384610 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5898814-d1b3-45a1-988b-4235504da6a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:49.385994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5898814-d1b3-45a1-988b-4235504da6a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:49.387519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5898814-d1b3-45a1-988b-4235504da6a7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:49.389716 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 70/142] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:20:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:49.394926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5898814-d1b3-45a1-988b-4235504da6a7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:49.415973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9764a1-f087-4e64-b204-56c526ace546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:49.417310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9764a1-f087-4e64-b204-56c526ace546 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:49.417749 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9764a1-f087-4e64-b204-56c526ace546 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:49.418710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9764a1-f087-4e64-b204-56c526ace546 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:49.435857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9764a1-f087-4e64-b204-56c526ace546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-8d9764a1-f087-4e64-b204-56c526ace546 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:49.448151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:49.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:49.450328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:49.450328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:49.450328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:49.450328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:49.450328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:49.450328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:49.450328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:49.450328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:49.450328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:49.450328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:49.450328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:20:49.450328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:20:49.452900 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 73/143] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:20:49.458752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9764a1-f087-4e64-b204-56c526ace546 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:49.523109 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5898814-d1b3-45a1-988b-4235504da6a7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:49.526561 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:49.564418 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8a689-e941-479f-bfde-c9300795f0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:49.565646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8a689-e941-479f-bfde-c9300795f0c4 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:49.566245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8a689-e941-479f-bfde-c9300795f0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:49.566757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8a689-e941-479f-bfde-c9300795f0c4 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:49.567694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8a689-e941-479f-bfde-c9300795f0c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:49.570375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8a689-e941-479f-bfde-c9300795f0c4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:49.620717 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:49.625992 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9764a1-f087-4e64-b204-56c526ace546 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:49.663031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b6b37-59ce-4fbb-be8e-3f5e05054efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:49.664164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b6b37-59ce-4fbb-be8e-3f5e05054efe None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:49.664655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b6b37-59ce-4fbb-be8e-3f5e05054efe None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:49.665213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b6b37-59ce-4fbb-be8e-3f5e05054efe None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:49.666236 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b6b37-59ce-4fbb-be8e-3f5e05054efe None admin] RBAC: Adding query filter params (name=reader) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:49.666766 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b6b37-59ce-4fbb-be8e-3f5e05054efe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:49.668475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b6b37-59ce-4fbb-be8e-3f5e05054efe None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:49.678216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 74/144] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:20:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:49.687088 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b6b37-59ce-4fbb-be8e-3f5e05054efe None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:49.709093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 71/145] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:20:49 2022] POST /identity/v3/users => generated 283 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:20:49.805714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b6b37-59ce-4fbb-be8e-3f5e05054efe None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:49.846501 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299c93d-a4e3-4ea9-810f-b37edb154c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:49.847441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299c93d-a4e3-4ea9-810f-b37edb154c5e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:49.847734 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299c93d-a4e3-4ea9-810f-b37edb154c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:49.848019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299c93d-a4e3-4ea9-810f-b37edb154c5e None admin] PATH_INFO: `/v3/users/38275baf180342959a6ee609014b2f1f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:49.849331 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299c93d-a4e3-4ea9-810f-b37edb154c5e None admin] RBAC: Authorizing `identity:get_user(user_id=38275baf180342959a6ee609014b2f1f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:49.851463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299c93d-a4e3-4ea9-810f-b37edb154c5e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:49.853294 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 75/146] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:49 2022] GET /identity/v3/users/38275baf180342959a6ee609014b2f1f => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:49.858972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8a689-e941-479f-bfde-c9300795f0c4 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:49.987335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8a689-e941-479f-bfde-c9300795f0c4 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:50.024426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a68e2d6-ccc5-4d6a-814d-502462ff913d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:50.025341 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68e2d6-ccc5-4d6a-814d-502462ff913d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:50.025634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68e2d6-ccc5-4d6a-814d-502462ff913d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:50.025920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68e2d6-ccc5-4d6a-814d-502462ff913d None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:50.027132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a68e2d6-ccc5-4d6a-814d-502462ff913d None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:50.028657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a68e2d6-ccc5-4d6a-814d-502462ff913d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:50.030478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 72/147] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:49 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:50.039768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299c93d-a4e3-4ea9-810f-b37edb154c5e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:50.071719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:50.148061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299c93d-a4e3-4ea9-810f-b37edb154c5e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:50.184863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95484364-1926-41c9-9548-47b4c3fe40fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:50.185812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95484364-1926-41c9-9548-47b4c3fe40fa None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:50.186102 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95484364-1926-41c9-9548-47b4c3fe40fa None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:50.186385 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95484364-1926-41c9-9548-47b4c3fe40fa None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b/users/38275baf180342959a6ee609014b2f1f/roles/1026857a0bd64410a463448b53532814` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:50.188407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95484364-1926-41c9-9548-47b4c3fe40fa None admin] RBAC: Authorizing `identity:create_grant(project_id=936684d8d665414698e461a87200ef9b, user_id=38275baf180342959a6ee609014b2f1f, role_id=1026857a0bd64410a463448b53532814)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:50.189957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95484364-1926-41c9-9548-47b4c3fe40fa None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:50.269887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 76/148] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:20:50 2022] PUT /identity/v3/projects/936684d8d665414698e461a87200ef9b/users/38275baf180342959a6ee609014b2f1f/roles/1026857a0bd64410a463448b53532814 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:20:51.318783 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708c2bc-47b9-47fc-829f-7b35f26bf363 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:51.319330 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708c2bc-47b9-47fc-829f-7b35f26bf363 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:51.319472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708c2bc-47b9-47fc-829f-7b35f26bf363 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:51.321832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 73/149] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:51.327041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef31fc1-7f71-4505-ba64-5b4b6b56af5c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:51.327460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef31fc1-7f71-4505-ba64-5b4b6b56af5c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:51.327650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef31fc1-7f71-4505-ba64-5b4b6b56af5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:51.349090 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fef31fc1-7f71-4505-ba64-5b4b6b56af5c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:51.350305 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fef31fc1-7f71-4505-ba64-5b4b6b56af5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:51.439430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 77/150] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:51.478903 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708c2bc-47b9-47fc-829f-7b35f26bf363 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:51.519978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:51.576558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601c06b-46cf-40ba-94a4-38e16cc44b0f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:51.576848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601c06b-46cf-40ba-94a4-38e16cc44b0f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:51.577049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601c06b-46cf-40ba-94a4-38e16cc44b0f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:51.578462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 78/151] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:20:51.603784 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2e0d2-085b-4525-9e14-d85f92583d13 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:51.604088 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2e0d2-085b-4525-9e14-d85f92583d13 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:51.604380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2e0d2-085b-4525-9e14-d85f92583d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:51.612869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708c2bc-47b9-47fc-829f-7b35f26bf363 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:51.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c3c2e0d2-085b-4525-9e14-d85f92583d13 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:51.641117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c3c2e0d2-085b-4525-9e14-d85f92583d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:51.662242 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:51.671671 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246681ee-4baf-42b6-885c-3d5617999762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:51.673266 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246681ee-4baf-42b6-885c-3d5617999762 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:51.673266 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246681ee-4baf-42b6-885c-3d5617999762 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:51.673679 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246681ee-4baf-42b6-885c-3d5617999762 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:51.697764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246681ee-4baf-42b6-885c-3d5617999762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-246681ee-4baf-42b6-885c-3d5617999762 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:51.706282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:51.707235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:51.708051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:51.708051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:51.708051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:51.708051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:51.708051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:51.708051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:51.708051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:51.708051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:51.708051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:51.708051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:51.708051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:51.708051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:20:51.708051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 74/152] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:20:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:51.713792 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246681ee-4baf-42b6-885c-3d5617999762 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:51.770446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 79/153] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:20:51.827964 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2e0d2-085b-4525-9e14-d85f92583d13 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:51.895782 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:51.917832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246681ee-4baf-42b6-885c-3d5617999762 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:51.973014 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d319a9f-3ac6-4eb9-90c7-7ddb72042211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:51.974133 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d319a9f-3ac6-4eb9-90c7-7ddb72042211 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:51.974569 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d319a9f-3ac6-4eb9-90c7-7ddb72042211 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:51.974991 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d319a9f-3ac6-4eb9-90c7-7ddb72042211 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:51.975978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d319a9f-3ac6-4eb9-90c7-7ddb72042211 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:51.976522 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d319a9f-3ac6-4eb9-90c7-7ddb72042211 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:51.977496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2e0d2-085b-4525-9e14-d85f92583d13 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:51.978127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d319a9f-3ac6-4eb9-90c7-7ddb72042211 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:51.990195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 75/154] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:51.996274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d319a9f-3ac6-4eb9-90c7-7ddb72042211 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:52.049817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561cea7-a85d-4a3d-8ea3-eed43a16eab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:52.050650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561cea7-a85d-4a3d-8ea3-eed43a16eab3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:52.050945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561cea7-a85d-4a3d-8ea3-eed43a16eab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:52.051229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561cea7-a85d-4a3d-8ea3-eed43a16eab3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:52.066781 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561cea7-a85d-4a3d-8ea3-eed43a16eab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-3561cea7-a85d-4a3d-8ea3-eed43a16eab3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:52.082915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:52.083908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:52.084712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:52.084712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:52.084712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:52.084712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:52.084712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:52.084712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:52.084712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:52.084712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:52.084712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:52.084712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:52.084712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:20:52.084712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:20:52.084712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 80/155] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:20:52.094229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561cea7-a85d-4a3d-8ea3-eed43a16eab3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:52.133462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:52.227535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d319a9f-3ac6-4eb9-90c7-7ddb72042211 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:52.265375 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561cea7-a85d-4a3d-8ea3-eed43a16eab3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:52.322744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468353e4-f992-4eec-91ac-0462a700bf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:52.323706 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468353e4-f992-4eec-91ac-0462a700bf01 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:52.324151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468353e4-f992-4eec-91ac-0462a700bf01 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:52.324588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468353e4-f992-4eec-91ac-0462a700bf01 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:52.324881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496c481-bad8-4c38-9fe2-f126742893ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:52.325536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468353e4-f992-4eec-91ac-0462a700bf01 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:52.325854 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468353e4-f992-4eec-91ac-0462a700bf01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:52.326617 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496c481-bad8-4c38-9fe2-f126742893ac None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:52.327058 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496c481-bad8-4c38-9fe2-f126742893ac None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:52.327334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468353e4-f992-4eec-91ac-0462a700bf01 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:52.327915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496c481-bad8-4c38-9fe2-f126742893ac None admin] PATH_INFO: `/v3/users/2781ce56481d4d179e62cc19ea41a8b0` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:52.333873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496c481-bad8-4c38-9fe2-f126742893ac None admin] RBAC: Authorizing `identity:get_user(user_id=2781ce56481d4d179e62cc19ea41a8b0)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:52.335728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496c481-bad8-4c38-9fe2-f126742893ac None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:52.338033 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 76/156] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:51 2022] GET /identity/v3/users/2781ce56481d4d179e62cc19ea41a8b0 => generated 247 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:52.340672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 81/157] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:20:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:52.343287 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496c481-bad8-4c38-9fe2-f126742893ac None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:52.349899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468353e4-f992-4eec-91ac-0462a700bf01 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:52.389419 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:52.469547 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:52.495738 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468353e4-f992-4eec-91ac-0462a700bf01 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:52.554259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7ab2c-cceb-4782-a7a2-4410375fee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:52.555366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7ab2c-cceb-4782-a7a2-4410375fee89 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:52.555658 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7ab2c-cceb-4782-a7a2-4410375fee89 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:52.555948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7ab2c-cceb-4782-a7a2-4410375fee89 None admin] PATH_INFO: `/v3/users/358e903ec0d14cd2809ef65ffb8035f5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:52.557963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7ab2c-cceb-4782-a7a2-4410375fee89 None admin] RBAC: Authorizing `identity:get_user(user_id=358e903ec0d14cd2809ef65ffb8035f5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:52.559465 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7ab2c-cceb-4782-a7a2-4410375fee89 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:52.561268 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 82/158] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:52 2022] GET /identity/v3/users/358e903ec0d14cd2809ef65ffb8035f5 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:52.562740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8197-56b4-495a-b4b1-9013ee519ee2 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:52.562949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8197-56b4-495a-b4b1-9013ee519ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:52.563247 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8197-56b4-495a-b4b1-9013ee519ee2 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:52.566445 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 83/159] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:52.567799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba8197-56b4-495a-b4b1-9013ee519ee2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:52.595180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496c481-bad8-4c38-9fe2-f126742893ac None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:52.680210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed4604-8ea7-485e-9586-5d7227a66606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:52.680843 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed4604-8ea7-485e-9586-5d7227a66606 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:52.681165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed4604-8ea7-485e-9586-5d7227a66606 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:52.681446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed4604-8ea7-485e-9586-5d7227a66606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:52.756174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba8197-56b4-495a-b4b1-9013ee519ee2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:52.785623 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed4604-8ea7-485e-9586-5d7227a66606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:52.787401 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed4604-8ea7-485e-9586-5d7227a66606 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:52.809747 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:52.832933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ce3f3-6858-4164-9b42-287f9fbfb914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:52.833638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ce3f3-6858-4164-9b42-287f9fbfb914 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:52.843739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ce3f3-6858-4164-9b42-287f9fbfb914 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:52.843739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ce3f3-6858-4164-9b42-287f9fbfb914 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:52.856046 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:52.907920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 77/160] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:20:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:20:52.910603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c1d5a-8609-4830-8e68-7541a825bfed None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:52.911807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c1d5a-8609-4830-8e68-7541a825bfed None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:52.912785 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c1d5a-8609-4830-8e68-7541a825bfed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:52.948005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-119c1d5a-8609-4830-8e68-7541a825bfed None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:52.949615 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-119c1d5a-8609-4830-8e68-7541a825bfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:52.983338 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ce3f3-6858-4164-9b42-287f9fbfb914 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:52.985018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ce3f3-6858-4164-9b42-287f9fbfb914 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:53.076728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 78/161] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:53.077871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c1d5a-8609-4830-8e68-7541a825bfed None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:53.187074 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 84/162] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:20:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:20:53.187482 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:53.188923 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ebcf5-c53c-40b5-95e9-05e7fbc5c6e4 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:53.193761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ebcf5-c53c-40b5-95e9-05e7fbc5c6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:53.194076 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ebcf5-c53c-40b5-95e9-05e7fbc5c6e4 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:53.196324 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 85/163] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:53.197251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ebcf5-c53c-40b5-95e9-05e7fbc5c6e4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:53.250390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:53.251986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c1d5a-8609-4830-8e68-7541a825bfed None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:53.290875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2f9f9-473c-4242-9c54-290b51aba800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:53.291977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f9f9-473c-4242-9c54-290b51aba800 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:53.292430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f9f9-473c-4242-9c54-290b51aba800 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:53.292847 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f9f9-473c-4242-9c54-290b51aba800 None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:53.294213 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2f9f9-473c-4242-9c54-290b51aba800 None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:53.295818 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2f9f9-473c-4242-9c54-290b51aba800 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:53.297702 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 79/164] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:53 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:53.298691 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2f9f9-473c-4242-9c54-290b51aba800 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:53.326514 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ebcf5-c53c-40b5-95e9-05e7fbc5c6e4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:53.394747 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef9f77-fd02-4784-af72-d98aa7749b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:53.395767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef9f77-fd02-4784-af72-d98aa7749b49 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:53.396187 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef9f77-fd02-4784-af72-d98aa7749b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:53.396569 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef9f77-fd02-4784-af72-d98aa7749b49 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:53.407748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2f9f9-473c-4242-9c54-290b51aba800 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:53.415593 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef9f77-fd02-4784-af72-d98aa7749b49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-7fef9f77-fd02-4784-af72-d98aa7749b49 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:53.424544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:53.425601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:53.426394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:53.426394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:53.426394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:53.426394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:53.426394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:53.426394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:53.426394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:53.426394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:53.426394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:53.426394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:53.426394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:20:53.426394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:20:53.430715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:53.431577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 86/165] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:20:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:53.432743 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2378b-a063-46ef-bc1a-b513507a0099 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:53.437160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2378b-a063-46ef-bc1a-b513507a0099 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:53.437160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2378b-a063-46ef-bc1a-b513507a0099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:53.461165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c3334-1af4-43e0-9d4f-59e8ee2b30ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:53.463250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c3334-1af4-43e0-9d4f-59e8ee2b30ec None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:53.464049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c3334-1af4-43e0-9d4f-59e8ee2b30ec None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:53.464832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c3334-1af4-43e0-9d4f-59e8ee2b30ec None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:53.466780 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c3334-1af4-43e0-9d4f-59e8ee2b30ec None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:53.468491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-63e2378b-a063-46ef-bc1a-b513507a0099 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:53.469675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c3334-1af4-43e0-9d4f-59e8ee2b30ec None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:53.469781 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-63e2378b-a063-46ef-bc1a-b513507a0099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:53.471735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 80/166] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:53 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:53.472758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c3334-1af4-43e0-9d4f-59e8ee2b30ec None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:53.585264 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c3334-1af4-43e0-9d4f-59e8ee2b30ec None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:53.594407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 87/167] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:53.595354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2378b-a063-46ef-bc1a-b513507a0099 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:53.623304 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad499d5-9bc6-495a-a099-72b45348a374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:53.624301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad499d5-9bc6-495a-a099-72b45348a374 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:53.624761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad499d5-9bc6-495a-a099-72b45348a374 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:53.625287 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad499d5-9bc6-495a-a099-72b45348a374 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:53.626427 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad499d5-9bc6-495a-a099-72b45348a374 None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=936684d8d665414698e461a87200ef9b, user.id=2781ce56481d4d179e62cc19ea41a8b0) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:53.626949 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad499d5-9bc6-495a-a099-72b45348a374 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:53.628766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad499d5-9bc6-495a-a099-72b45348a374 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:53.636287 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 81/168] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:20:53 2022] GET /identity/v3/role_assignments?user.id=2781ce56481d4d179e62cc19ea41a8b0&scope.project.id=936684d8d665414698e461a87200ef9b&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:53.637421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad499d5-9bc6-495a-a099-72b45348a374 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:53.650213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:53.692747 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:53.751655 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad499d5-9bc6-495a-a099-72b45348a374 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:53.789680 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ebdf6-2b39-4a0d-944f-3f23f8203808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:53.790731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ebdf6-2b39-4a0d-944f-3f23f8203808 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:53.791214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ebdf6-2b39-4a0d-944f-3f23f8203808 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:53.791707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ebdf6-2b39-4a0d-944f-3f23f8203808 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:53.792779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ebdf6-2b39-4a0d-944f-3f23f8203808 None admin] RBAC: Adding query filter params (role.id=9330473f62ec4a82a82d1c4beb6160d7, scope.project.id=936684d8d665414698e461a87200ef9b, user.id=358e903ec0d14cd2809ef65ffb8035f5) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:53.793325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ebdf6-2b39-4a0d-944f-3f23f8203808 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:53.795019 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ebdf6-2b39-4a0d-944f-3f23f8203808 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:53.802250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 82/169] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:20:53 2022] GET /identity/v3/role_assignments?user.id=358e903ec0d14cd2809ef65ffb8035f5&scope.project.id=936684d8d665414698e461a87200ef9b&role.id=9330473f62ec4a82a82d1c4beb6160d7 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:53.803921 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ebdf6-2b39-4a0d-944f-3f23f8203808 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:53.835398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2378b-a063-46ef-bc1a-b513507a0099 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:53.936728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72156ce0-cc6c-4db0-ad24-09753fd7123c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:53.937441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72156ce0-cc6c-4db0-ad24-09753fd7123c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:53.937728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72156ce0-cc6c-4db0-ad24-09753fd7123c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:53.938002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72156ce0-cc6c-4db0-ad24-09753fd7123c None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:53.938841 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72156ce0-cc6c-4db0-ad24-09753fd7123c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:53.939053 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72156ce0-cc6c-4db0-ad24-09753fd7123c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:53.946064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72156ce0-cc6c-4db0-ad24-09753fd7123c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:53.959977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 88/170] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:20:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:53.961802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ebdf6-2b39-4a0d-944f-3f23f8203808 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:53.969167 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72156ce0-cc6c-4db0-ad24-09753fd7123c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:53.982800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:54.006062 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b74b6-33d4-43ef-b4f4-3fa5ccfded34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:54.008071 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b74b6-33d4-43ef-b4f4-3fa5ccfded34 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:54.008603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b74b6-33d4-43ef-b4f4-3fa5ccfded34 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:54.009162 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b74b6-33d4-43ef-b4f4-3fa5ccfded34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:54.023547 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b74b6-33d4-43ef-b4f4-3fa5ccfded34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-950b74b6-33d4-43ef-b4f4-3fa5ccfded34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:54.031871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:54.032821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:54.033672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:54.033672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:54.033672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:54.033672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:54.033672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:54.033672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:54.033672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:54.033672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:54.033672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:54.033672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:54.033672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:20:54.033672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:20:54.036359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 83/171] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:54.048036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b74b6-33d4-43ef-b4f4-3fa5ccfded34 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:54.176035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:54.180216 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b74b6-33d4-43ef-b4f4-3fa5ccfded34 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:54.193656 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72156ce0-cc6c-4db0-ad24-09753fd7123c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:54.219211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577a5f6-af2e-44ce-8701-43adbb06e196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:54.220231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577a5f6-af2e-44ce-8701-43adbb06e196 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:54.220724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577a5f6-af2e-44ce-8701-43adbb06e196 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:54.221236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577a5f6-af2e-44ce-8701-43adbb06e196 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:54.222298 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577a5f6-af2e-44ce-8701-43adbb06e196 None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:54.222820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577a5f6-af2e-44ce-8701-43adbb06e196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:54.224504 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577a5f6-af2e-44ce-8701-43adbb06e196 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:54.234509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 84/172] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:20:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:54.240985 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577a5f6-af2e-44ce-8701-43adbb06e196 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:54.271744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427158e-bcbe-49f3-8a62-97dd5803b391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:54.273086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427158e-bcbe-49f3-8a62-97dd5803b391 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:54.274024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427158e-bcbe-49f3-8a62-97dd5803b391 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:54.275581 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427158e-bcbe-49f3-8a62-97dd5803b391 None admin] PATH_INFO: `/v3/users/358e903ec0d14cd2809ef65ffb8035f5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:54.278124 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427158e-bcbe-49f3-8a62-97dd5803b391 None admin] RBAC: Authorizing `identity:get_user(user_id=358e903ec0d14cd2809ef65ffb8035f5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:54.282052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427158e-bcbe-49f3-8a62-97dd5803b391 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:54.285119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 89/173] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:53 2022] GET /identity/v3/users/358e903ec0d14cd2809ef65ffb8035f5 => generated 275 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:54.293311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427158e-bcbe-49f3-8a62-97dd5803b391 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:54.317015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:54.447913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577a5f6-af2e-44ce-8701-43adbb06e196 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:54.493728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427158e-bcbe-49f3-8a62-97dd5803b391 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:54.518323 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06943a62-8c69-4bc5-825b-1442421f5c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:54.519237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06943a62-8c69-4bc5-825b-1442421f5c92 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:54.519520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06943a62-8c69-4bc5-825b-1442421f5c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:54.519804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06943a62-8c69-4bc5-825b-1442421f5c92 None admin] PATH_INFO: `/v3/users/358e903ec0d14cd2809ef65ffb8035f5` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:54.521036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06943a62-8c69-4bc5-825b-1442421f5c92 None admin] RBAC: Authorizing `identity:get_user(user_id=358e903ec0d14cd2809ef65ffb8035f5)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:54.522585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06943a62-8c69-4bc5-825b-1442421f5c92 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:54.524552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 85/174] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:54 2022] GET /identity/v3/users/358e903ec0d14cd2809ef65ffb8035f5 => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:54.527403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9354004-daf6-499f-9414-6ef00c064522 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:54.527403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9354004-daf6-499f-9414-6ef00c064522 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:54.527403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9354004-daf6-499f-9414-6ef00c064522 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:54.527648 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 86/175] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:54.535158 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83d540-6e7f-4482-aac7-e2d41a175ce4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:54.535451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83d540-6e7f-4482-aac7-e2d41a175ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:54.535736 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83d540-6e7f-4482-aac7-e2d41a175ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:54.550993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412b631-e004-4b35-8585-d32902c68568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:54.552091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412b631-e004-4b35-8585-d32902c68568 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:54.552614 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412b631-e004-4b35-8585-d32902c68568 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:54.553113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412b631-e004-4b35-8585-d32902c68568 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:54.568372 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-bc83d540-6e7f-4482-aac7-e2d41a175ce4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:54.568372 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bc83d540-6e7f-4482-aac7-e2d41a175ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:54.591807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:54.649742 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412b631-e004-4b35-8585-d32902c68568 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:54.651285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412b631-e004-4b35-8585-d32902c68568 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:54.737097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 87/176] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:20:54.737981 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83d540-6e7f-4482-aac7-e2d41a175ce4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:54.792405 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:54.844473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 90/177] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:20:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:20:54.851519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac9f6f-d35f-4848-93dd-d3b95bbbdb3c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:54.852106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac9f6f-d35f-4848-93dd-d3b95bbbdb3c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:54.852580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac9f6f-d35f-4848-93dd-d3b95bbbdb3c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:54.854216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 91/178] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:54.855155 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac9f6f-d35f-4848-93dd-d3b95bbbdb3c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:54.871094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83d540-6e7f-4482-aac7-e2d41a175ce4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:54.915950 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ebeba-7b23-474d-9c8b-10b426023cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:54.917475 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ebeba-7b23-474d-9c8b-10b426023cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:54.917967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ebeba-7b23-474d-9c8b-10b426023cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:54.918854 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ebeba-7b23-474d-9c8b-10b426023cf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:54.925302 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:54.968472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac9f6f-d35f-4848-93dd-d3b95bbbdb3c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:55.007360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ebeba-7b23-474d-9c8b-10b426023cf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:55.008376 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05bbc0-e497-4cc7-a8fd-6ab3ee59db75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:55.009227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05bbc0-e497-4cc7-a8fd-6ab3ee59db75 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:55.009513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05bbc0-e497-4cc7-a8fd-6ab3ee59db75 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:55.009865 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05bbc0-e497-4cc7-a8fd-6ab3ee59db75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:55.010191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ebeba-7b23-474d-9c8b-10b426023cf3 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:55.022192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05bbc0-e497-4cc7-a8fd-6ab3ee59db75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-ac05bbc0-e497-4cc7-a8fd-6ab3ee59db75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:55.031091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:55.032081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:55.032904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:55.032904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:55.032904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:55.032904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:55.032904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:55.032904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:55.032904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:55.032904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:55.032904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:55.032904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:55.032904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:55.032904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:20:55.032904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 92/179] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:20:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:55.039504 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05bbc0-e497-4cc7-a8fd-6ab3ee59db75 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:55.055963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:55.132897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 88/180] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:20:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:20:55.134681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81003a55-c860-4db1-be2f-a8804f3bd4a8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:55.135188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81003a55-c860-4db1-be2f-a8804f3bd4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:55.135678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81003a55-c860-4db1-be2f-a8804f3bd4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:55.145345 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05bbc0-e497-4cc7-a8fd-6ab3ee59db75 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:55.154955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-81003a55-c860-4db1-be2f-a8804f3bd4a8 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:55.156547 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-81003a55-c860-4db1-be2f-a8804f3bd4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:55.168079 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:55.188573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261bf10-0e9e-445c-8d30-c9267f380c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:55.189541 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261bf10-0e9e-445c-8d30-c9267f380c58 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:55.189768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261bf10-0e9e-445c-8d30-c9267f380c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:55.190048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261bf10-0e9e-445c-8d30-c9267f380c58 None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:55.191243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261bf10-0e9e-445c-8d30-c9267f380c58 None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:55.192896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261bf10-0e9e-445c-8d30-c9267f380c58 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:55.199617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 93/181] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:55 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:55.199617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261bf10-0e9e-445c-8d30-c9267f380c58 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:55.236982 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 89/182] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:55.237898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81003a55-c860-4db1-be2f-a8804f3bd4a8 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:55.315678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:55.356336 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81003a55-c860-4db1-be2f-a8804f3bd4a8 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:55.371978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261bf10-0e9e-445c-8d30-c9267f380c58 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:55.407154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e82a78-56ec-4c36-8abc-81edceec84d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:55.408043 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e82a78-56ec-4c36-8abc-81edceec84d8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:55.408336 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e82a78-56ec-4c36-8abc-81edceec84d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:55.408621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e82a78-56ec-4c36-8abc-81edceec84d8 None admin] PATH_INFO: `/v3/projects/5575c0a712014ebc84e5feaed1ff36df` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:55.409885 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e82a78-56ec-4c36-8abc-81edceec84d8 None admin] RBAC: Authorizing `identity:get_project(project_id=5575c0a712014ebc84e5feaed1ff36df)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:55.411352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e82a78-56ec-4c36-8abc-81edceec84d8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:55.413399 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 90/183] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:55 2022] GET /identity/v3/projects/5575c0a712014ebc84e5feaed1ff36df => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:55.414076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e82a78-56ec-4c36-8abc-81edceec84d8 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:55.420110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c700-a665-4406-a7a0-45924b8c9175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:55.421204 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c700-a665-4406-a7a0-45924b8c9175 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:55.421693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c700-a665-4406-a7a0-45924b8c9175 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:55.422225 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c700-a665-4406-a7a0-45924b8c9175 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:55.424193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c700-a665-4406-a7a0-45924b8c9175 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:55.424733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c700-a665-4406-a7a0-45924b8c9175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:55.426685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c700-a665-4406-a7a0-45924b8c9175 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:55.442272 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 94/184] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:55.442272 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894b822-3ca8-4f41-b233-993b1c76a314 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:55.442272 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894b822-3ca8-4f41-b233-993b1c76a314 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:55.442272 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894b822-3ca8-4f41-b233-993b1c76a314 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:55.443164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 95/185] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:55.445018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894b822-3ca8-4f41-b233-993b1c76a314 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:55.513740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:55.524521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e82a78-56ec-4c36-8abc-81edceec84d8 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:55.555927 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:55.562758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b50bce-7475-447e-a4f7-83088b3ed890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:55.563560 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b50bce-7475-447e-a4f7-83088b3ed890 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:55.570413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b50bce-7475-447e-a4f7-83088b3ed890 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:55.570413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b50bce-7475-447e-a4f7-83088b3ed890 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:55.570413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b50bce-7475-447e-a4f7-83088b3ed890 None admin] RBAC: Adding query filter params (role.id=1bac0428b2544e2cb16fad539f080c14, scope.project.id=936684d8d665414698e461a87200ef9b, user.id=358e903ec0d14cd2809ef65ffb8035f5) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:55.570413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b50bce-7475-447e-a4f7-83088b3ed890 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:55.570413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b50bce-7475-447e-a4f7-83088b3ed890 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:55.570847 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894b822-3ca8-4f41-b233-993b1c76a314 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:55.575480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 91/186] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:20:55 2022] GET /identity/v3/role_assignments?user.id=358e903ec0d14cd2809ef65ffb8035f5&scope.project.id=936684d8d665414698e461a87200ef9b&role.id=1bac0428b2544e2cb16fad539f080c14 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:55.576987 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc95f0-ce54-4a06-b2dd-610065ee83d4 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:55.577310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc95f0-ce54-4a06-b2dd-610065ee83d4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:55.577592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc95f0-ce54-4a06-b2dd-610065ee83d4 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:55.579015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 92/187] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:55.584655 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc95f0-ce54-4a06-b2dd-610065ee83d4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:55.625645 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fe62d-7c50-4344-99a9-78a85fc9039a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:55.627691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fe62d-7c50-4344-99a9-78a85fc9039a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:55.628198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fe62d-7c50-4344-99a9-78a85fc9039a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:55.628691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fe62d-7c50-4344-99a9-78a85fc9039a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:55.648420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93fe62d-7c50-4344-99a9-78a85fc9039a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d93fe62d-7c50-4344-99a9-78a85fc9039a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:55.659065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:55.660153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:55.660956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:55.660956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:55.660956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:55.660956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:55.660956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:55.660956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:55.660956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:55.660956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:55.660956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:55.660956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:55.660956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:20:55.660956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:20:55.660956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 96/188] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:55.661690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fe62d-7c50-4344-99a9-78a85fc9039a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:55.722027 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc95f0-ce54-4a06-b2dd-610065ee83d4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:55.767297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fb36a-ee4c-4d17-ada2-76b8df9675ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:55.767977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fb36a-ee4c-4d17-ada2-76b8df9675ad None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:55.768279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fb36a-ee4c-4d17-ada2-76b8df9675ad None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:55.768550 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fb36a-ee4c-4d17-ada2-76b8df9675ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:55.769464 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fb36a-ee4c-4d17-ada2-76b8df9675ad None admin] RBAC: Adding query filter params (role.id=9330473f62ec4a82a82d1c4beb6160d7, scope.project.id=5575c0a712014ebc84e5feaed1ff36df, user.id=358e903ec0d14cd2809ef65ffb8035f5) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:55.769738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fb36a-ee4c-4d17-ada2-76b8df9675ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:55.771220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fb36a-ee4c-4d17-ada2-76b8df9675ad None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:55.777675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 93/189] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:20:55 2022] GET /identity/v3/role_assignments?user.id=358e903ec0d14cd2809ef65ffb8035f5&scope.project.id=5575c0a712014ebc84e5feaed1ff36df&role.id=9330473f62ec4a82a82d1c4beb6160d7 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:55.777675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588679c-bbd7-43d1-85df-6a01dd6f32b5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:55.777675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588679c-bbd7-43d1-85df-6a01dd6f32b5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:55.777675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588679c-bbd7-43d1-85df-6a01dd6f32b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:55.803981 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2588679c-bbd7-43d1-85df-6a01dd6f32b5 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:55.805833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2588679c-bbd7-43d1-85df-6a01dd6f32b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:55.822797 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fe62d-7c50-4344-99a9-78a85fc9039a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:55.843607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:55.876599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:55.890824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b9631-a836-485e-b553-8fb62a94f452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:55.892161 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b9631-a836-485e-b553-8fb62a94f452 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:55.892664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b9631-a836-485e-b553-8fb62a94f452 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:55.893237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b9631-a836-485e-b553-8fb62a94f452 None admin] PATH_INFO: `/v3/users/9447d090257c48fbb6343720d2916622` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:55.896147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 94/190] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:55.898668 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae1a95-c3d2-4f08-9f46-ac950b7ee3be None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:55.898968 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae1a95-c3d2-4f08-9f46-ac950b7ee3be None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:55.899255 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae1a95-c3d2-4f08-9f46-ac950b7ee3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:55.914535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b9631-a836-485e-b553-8fb62a94f452 None admin] RBAC: Authorizing `identity:get_user(user_id=9447d090257c48fbb6343720d2916622)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:55.916713 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b9631-a836-485e-b553-8fb62a94f452 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:55.919249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-12ae1a95-c3d2-4f08-9f46-ac950b7ee3be None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:55.919795 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 97/191] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:55 2022] GET /identity/v3/users/9447d090257c48fbb6343720d2916622 => generated 283 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:55.920896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-12ae1a95-c3d2-4f08-9f46-ac950b7ee3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:55.921344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b9631-a836-485e-b553-8fb62a94f452 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:56.002216 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 95/192] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:56.003302 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae1a95-c3d2-4f08-9f46-ac950b7ee3be None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:56.069700 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b9631-a836-485e-b553-8fb62a94f452 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:56.110919 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b3e8d-6612-4475-94f6-476872ec7ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:56.111885 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b3e8d-6612-4475-94f6-476872ec7ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:56.112371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b3e8d-6612-4475-94f6-476872ec7ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:56.112846 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b3e8d-6612-4475-94f6-476872ec7ff5 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:56.114802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b3e8d-6612-4475-94f6-476872ec7ff5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:56.115339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b3e8d-6612-4475-94f6-476872ec7ff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:56.117515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b3e8d-6612-4475-94f6-476872ec7ff5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:56.118337 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae1a95-c3d2-4f08-9f46-ac950b7ee3be None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:56.132446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 98/193] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:20:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:56.132446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b3e8d-6612-4475-94f6-476872ec7ff5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:56.136773 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:56.154908 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc3505-5900-4b11-9b47-193217f46c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:56.155570 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc3505-5900-4b11-9b47-193217f46c41 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:56.155860 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc3505-5900-4b11-9b47-193217f46c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:56.156146 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc3505-5900-4b11-9b47-193217f46c41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:56.162765 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:56.224167 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc3505-5900-4b11-9b47-193217f46c41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:56.226341 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc3505-5900-4b11-9b47-193217f46c41 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:56.233485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b3e8d-6612-4475-94f6-476872ec7ff5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:56.266534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ef448-fa51-4ef8-abb5-1c3766f5f059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:56.267342 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ef448-fa51-4ef8-abb5-1c3766f5f059 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:56.267626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ef448-fa51-4ef8-abb5-1c3766f5f059 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:56.267905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ef448-fa51-4ef8-abb5-1c3766f5f059 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:56.279022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ef448-fa51-4ef8-abb5-1c3766f5f059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-823ef448-fa51-4ef8-abb5-1c3766f5f059 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:56.287017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:56.288213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:56.289275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:56.289275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:56.289275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:56.289275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:56.289275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:56.289275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:56.289275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:56.289275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:56.289275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:56.289275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:56.289275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:20:56.289275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:20:56.289275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 99/194] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:20:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:56.289996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ef448-fa51-4ef8-abb5-1c3766f5f059 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:56.326572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 96/195] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:20:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:20:56.327710 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc3505-5900-4b11-9b47-193217f46c41 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:56.346889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:56.381113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:56.393039 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ef448-fa51-4ef8-abb5-1c3766f5f059 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:56.426618 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fd6fc-4329-4959-9748-385421bec7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:56.427416 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fd6fc-4329-4959-9748-385421bec7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:56.427702 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc3505-5900-4b11-9b47-193217f46c41 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:56.427812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fd6fc-4329-4959-9748-385421bec7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:56.427981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fd6fc-4329-4959-9748-385421bec7e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:56.429552 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fd6fc-4329-4959-9748-385421bec7e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:56.431006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fd6fc-4329-4959-9748-385421bec7e0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:56.432633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 100/196] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:20:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:56.434643 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fd6fc-4329-4959-9748-385421bec7e0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:56.461909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb1d83-7de2-4073-9e9b-9ea8df1538fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:56.462871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb1d83-7de2-4073-9e9b-9ea8df1538fe None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:56.463230 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb1d83-7de2-4073-9e9b-9ea8df1538fe None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:56.463586 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb1d83-7de2-4073-9e9b-9ea8df1538fe None admin] PATH_INFO: `/v3/users/38275baf180342959a6ee609014b2f1f` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:56.464788 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb1d83-7de2-4073-9e9b-9ea8df1538fe None admin] RBAC: Authorizing `identity:get_user(user_id=38275baf180342959a6ee609014b2f1f)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:56.466274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb1d83-7de2-4073-9e9b-9ea8df1538fe None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:56.467902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 97/197] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:56 2022] GET /identity/v3/users/38275baf180342959a6ee609014b2f1f => generated 289 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:56.468673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb1d83-7de2-4073-9e9b-9ea8df1538fe None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:56.532092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fd6fc-4329-4959-9748-385421bec7e0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:56.554251 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:56.565626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21102ea8-9c2e-42d6-b064-b3ae0914e344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:56.566379 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21102ea8-9c2e-42d6-b064-b3ae0914e344 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:56.566746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21102ea8-9c2e-42d6-b064-b3ae0914e344 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:56.567105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21102ea8-9c2e-42d6-b064-b3ae0914e344 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:56.567860 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21102ea8-9c2e-42d6-b064-b3ae0914e344 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:56.568176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21102ea8-9c2e-42d6-b064-b3ae0914e344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:56.568488 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb1d83-7de2-4073-9e9b-9ea8df1538fe None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:56.569797 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21102ea8-9c2e-42d6-b064-b3ae0914e344 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:56.577722 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 101/198] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:20:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:56.578507 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21102ea8-9c2e-42d6-b064-b3ae0914e344 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:56.599041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:56.603271 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de8bae-748d-479f-b0c3-ac9500672c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:56.604134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de8bae-748d-479f-b0c3-ac9500672c3c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:56.604416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de8bae-748d-479f-b0c3-ac9500672c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:56.604694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de8bae-748d-479f-b0c3-ac9500672c3c None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:56.605898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de8bae-748d-479f-b0c3-ac9500672c3c None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:56.607382 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de8bae-748d-479f-b0c3-ac9500672c3c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:56.609089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 98/199] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:56 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:56.609853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de8bae-748d-479f-b0c3-ac9500672c3c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:56.683107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21102ea8-9c2e-42d6-b064-b3ae0914e344 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:56.706449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de8bae-748d-479f-b0c3-ac9500672c3c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:56.721917 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41888d19-9d4e-48d0-9aa1-4fd9a5e3bdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:56.722599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41888d19-9d4e-48d0-9aa1-4fd9a5e3bdd1 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:56.722894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41888d19-9d4e-48d0-9aa1-4fd9a5e3bdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:56.723177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41888d19-9d4e-48d0-9aa1-4fd9a5e3bdd1 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:56.723906 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41888d19-9d4e-48d0-9aa1-4fd9a5e3bdd1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:56.725995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41888d19-9d4e-48d0-9aa1-4fd9a5e3bdd1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:56.740722 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1dc18-f655-4826-ab20-8ac47001ad3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:56.741448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1dc18-f655-4826-ab20-8ac47001ad3c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:56.741737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1dc18-f655-4826-ab20-8ac47001ad3c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:56.742020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1dc18-f655-4826-ab20-8ac47001ad3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:56.765715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:56.808273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1dc18-f655-4826-ab20-8ac47001ad3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:56.809779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1dc18-f655-4826-ab20-8ac47001ad3c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:56.822975 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 102/200] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:20:56 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:20:56.822975 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41888d19-9d4e-48d0-9aa1-4fd9a5e3bdd1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:56.923823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 99/201] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:20:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:20:56.924849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1dc18-f655-4826-ab20-8ac47001ad3c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:56.929731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:56.960927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41888d19-9d4e-48d0-9aa1-4fd9a5e3bdd1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:56.987360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:57.000139 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fc270-26ea-4fcc-be08-e31c09ccdc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:57.001039 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fc270-26ea-4fcc-be08-e31c09ccdc8d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:57.001364 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fc270-26ea-4fcc-be08-e31c09ccdc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:57.001669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fc270-26ea-4fcc-be08-e31c09ccdc8d None admin] PATH_INFO: `/v3/users/9447d090257c48fbb6343720d2916622` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:57.003051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fc270-26ea-4fcc-be08-e31c09ccdc8d None admin] RBAC: Authorizing `identity:get_user(user_id=9447d090257c48fbb6343720d2916622)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:57.004575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fc270-26ea-4fcc-be08-e31c09ccdc8d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:57.006356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 103/202] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:56 2022] GET /identity/v3/users/9447d090257c48fbb6343720d2916622 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:57.007187 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fc270-26ea-4fcc-be08-e31c09ccdc8d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:57.040759 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1dc18-f655-4826-ab20-8ac47001ad3c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:57.094887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9354f5-33a2-4116-91ed-cc2c47723b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:57.095945 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9354f5-33a2-4116-91ed-cc2c47723b5e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:57.096539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9354f5-33a2-4116-91ed-cc2c47723b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:57.096996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9354f5-33a2-4116-91ed-cc2c47723b5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:57.098850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9354f5-33a2-4116-91ed-cc2c47723b5e None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=bc8a10b282fb412fbc79062901f84a10, user.id=9447d090257c48fbb6343720d2916622) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:57.099377 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9354f5-33a2-4116-91ed-cc2c47723b5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:57.101883 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9354f5-33a2-4116-91ed-cc2c47723b5e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:57.110499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 100/203] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:20:56 2022] GET /identity/v3/role_assignments?user.id=9447d090257c48fbb6343720d2916622&scope.project.id=bc8a10b282fb412fbc79062901f84a10&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:57.111223 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9354f5-33a2-4116-91ed-cc2c47723b5e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:57.145950 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fc270-26ea-4fcc-be08-e31c09ccdc8d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:57.191907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa2ff9-7c39-4814-9265-47cf1288f8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:57.192875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa2ff9-7c39-4814-9265-47cf1288f8cd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:57.193268 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa2ff9-7c39-4814-9265-47cf1288f8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:57.193604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa2ff9-7c39-4814-9265-47cf1288f8cd None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:57.196261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa2ff9-7c39-4814-9265-47cf1288f8cd None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:57.197790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa2ff9-7c39-4814-9265-47cf1288f8cd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:57.199666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 104/204] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:57 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:57.205715 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa2ff9-7c39-4814-9265-47cf1288f8cd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:57.224609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:57.265000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9354f5-33a2-4116-91ed-cc2c47723b5e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:57.286160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:57.309031 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8a395-7647-4287-ac3e-9c64f0000c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:57.309764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8a395-7647-4287-ac3e-9c64f0000c5e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:57.310053 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8a395-7647-4287-ac3e-9c64f0000c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:57.310331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8a395-7647-4287-ac3e-9c64f0000c5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:57.364004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa2ff9-7c39-4814-9265-47cf1288f8cd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:57.432266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b02db-9553-4502-ae89-432fdbfd5e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:57.432939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b02db-9553-4502-ae89-432fdbfd5e0f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:57.433260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b02db-9553-4502-ae89-432fdbfd5e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:57.433545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b02db-9553-4502-ae89-432fdbfd5e0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:57.435630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b02db-9553-4502-ae89-432fdbfd5e0f None admin] RBAC: Adding query filter params (role.id=1026857a0bd64410a463448b53532814, scope.project.id=936684d8d665414698e461a87200ef9b, user.id=38275baf180342959a6ee609014b2f1f) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:57.435962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b02db-9553-4502-ae89-432fdbfd5e0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:57.437456 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b02db-9553-4502-ae89-432fdbfd5e0f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:57.440135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8a395-7647-4287-ac3e-9c64f0000c5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:57.441703 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8a395-7647-4287-ac3e-9c64f0000c5e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:57.454236 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 105/205] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:20:57 2022] GET /identity/v3/role_assignments?user.id=38275baf180342959a6ee609014b2f1f&scope.project.id=936684d8d665414698e461a87200ef9b&role.id=1026857a0bd64410a463448b53532814 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:20:57.598335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 101/206] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:20:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:20:57.604127 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b02db-9553-4502-ae89-432fdbfd5e0f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:57.719662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:57.748638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b02db-9553-4502-ae89-432fdbfd5e0f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:57.786131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df2d67-9a15-422d-ae2c-a55c2c7d136f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:57.787004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df2d67-9a15-422d-ae2c-a55c2c7d136f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:57.787292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df2d67-9a15-422d-ae2c-a55c2c7d136f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:57.787622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df2d67-9a15-422d-ae2c-a55c2c7d136f None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:57.789908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df2d67-9a15-422d-ae2c-a55c2c7d136f None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:57.791378 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df2d67-9a15-422d-ae2c-a55c2c7d136f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:57.793434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 106/207] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:20:57 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:57.798828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8a395-7647-4287-ac3e-9c64f0000c5e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:57.823018 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:57.914916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8a395-7647-4287-ac3e-9c64f0000c5e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:57.953348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43239a3d-a477-4add-aaf3-9ec1aaf0fab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:57.954020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43239a3d-a477-4add-aaf3-9ec1aaf0fab1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:57.954314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43239a3d-a477-4add-aaf3-9ec1aaf0fab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:57.954601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43239a3d-a477-4add-aaf3-9ec1aaf0fab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:57.955452 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43239a3d-a477-4add-aaf3-9ec1aaf0fab1 None admin] RBAC: Adding query filter params (role.id=1bac0428b2544e2cb16fad539f080c14, scope.project.id=bc8a10b282fb412fbc79062901f84a10, user.id=9447d090257c48fbb6343720d2916622) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:57.955772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43239a3d-a477-4add-aaf3-9ec1aaf0fab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:57.957263 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43239a3d-a477-4add-aaf3-9ec1aaf0fab1 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:57.964794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 102/208] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:20:57 2022] GET /identity/v3/role_assignments?user.id=9447d090257c48fbb6343720d2916622&scope.project.id=bc8a10b282fb412fbc79062901f84a10&role.id=1bac0428b2544e2cb16fad539f080c14 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:59.305757 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64541b-0ee5-4a42-be9f-812e5af8b399 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:59.305757 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64541b-0ee5-4a42-be9f-812e5af8b399 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:59.305757 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64541b-0ee5-4a42-be9f-812e5af8b399 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:59.306893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 107/209] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:59.313020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4f5a5-7921-4429-8360-92da0166b6e9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:59.313861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4f5a5-7921-4429-8360-92da0166b6e9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:59.314355 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4f5a5-7921-4429-8360-92da0166b6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:59.332196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3fe4f5a5-7921-4429-8360-92da0166b6e9 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:59.333752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3fe4f5a5-7921-4429-8360-92da0166b6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:59.408019 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 103/210] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:59.431923 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64541b-0ee5-4a42-be9f-812e5af8b399 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:59.476469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cab70-9d93-4288-b631-12dc4ca1f646 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:59.476774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cab70-9d93-4288-b631-12dc4ca1f646 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:59.477049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cab70-9d93-4288-b631-12dc4ca1f646 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:59.478640 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 104/211] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:59.484481 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aecfc0-f53e-426c-931b-4172ee11bc6d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:59.484773 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aecfc0-f53e-426c-931b-4172ee11bc6d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:59.485080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aecfc0-f53e-426c-931b-4172ee11bc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:59.508068 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-73aecfc0-f53e-426c-931b-4172ee11bc6d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:59.509824 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-73aecfc0-f53e-426c-931b-4172ee11bc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:59.520266 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:59.553391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64541b-0ee5-4a42-be9f-812e5af8b399 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:59.593737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 105/212] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:20:59.595008 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d1187-ccc9-4d10-8f5d-89bea238fb46 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:59.595304 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d1187-ccc9-4d10-8f5d-89bea238fb46 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:59.595585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d1187-ccc9-4d10-8f5d-89bea238fb46 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:59.596975 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 106/213] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:20:59.600969 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccf0d1-c18a-48fa-9a28-2a836c064164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:59.602100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccf0d1-c18a-48fa-9a28-2a836c064164 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:59.602559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccf0d1-c18a-48fa-9a28-2a836c064164 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:59.603007 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccf0d1-c18a-48fa-9a28-2a836c064164 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:59.603464 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d2e68-8ded-4ed5-9cc6-bb22c777ec46 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:59.603763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d2e68-8ded-4ed5-9cc6-bb22c777ec46 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:59.604052 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d2e68-8ded-4ed5-9cc6-bb22c777ec46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:59.614453 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ccf0d1-c18a-48fa-9a28-2a836c064164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:59.622491 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ff2d2e68-8ded-4ed5-9cc6-bb22c777ec46 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-80ccf0d1-c18a-48fa-9a28-2a836c064164 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:59.622796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:59.627791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:59.628609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:59.628609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:59.628609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:59.628609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:59.628609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:59.628609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:59.628609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:59.628609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:59.628609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:59.628609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:59.628609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:20:59.628609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:20:59.629267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ff2d2e68-8ded-4ed5-9cc6-bb22c777ec46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:20:59.629338 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 108/214] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:20:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:59.629338 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccf0d1-c18a-48fa-9a28-2a836c064164 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:59.631897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:59.703343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 107/215] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:20:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:20:59.704213 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d2e68-8ded-4ed5-9cc6-bb22c777ec46 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:59.742026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccf0d1-c18a-48fa-9a28-2a836c064164 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:59.776939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6f10e-cce1-4a59-89d7-2e597741928c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:59.777766 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6f10e-cce1-4a59-89d7-2e597741928c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:59.778292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6f10e-cce1-4a59-89d7-2e597741928c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:59.778292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6f10e-cce1-4a59-89d7-2e597741928c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:59.779532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6f10e-cce1-4a59-89d7-2e597741928c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:59.780915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6f10e-cce1-4a59-89d7-2e597741928c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:59.786908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 109/216] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:20:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:59.786908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6f10e-cce1-4a59-89d7-2e597741928c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:59.794531 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:59.823710 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d2e68-8ded-4ed5-9cc6-bb22c777ec46 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:59.860141 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3b59b-3a8f-4f5a-85b3-aa9cfb5479cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:59.860733 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3b59b-3a8f-4f5a-85b3-aa9cfb5479cb None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:59.861050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3b59b-3a8f-4f5a-85b3-aa9cfb5479cb None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:59.861324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3b59b-3a8f-4f5a-85b3-aa9cfb5479cb None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:59.861701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:20:59.862067 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3b59b-3a8f-4f5a-85b3-aa9cfb5479cb None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:20:59.862378 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3b59b-3a8f-4f5a-85b3-aa9cfb5479cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:20:59.864248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3b59b-3a8f-4f5a-85b3-aa9cfb5479cb None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:20:59.873539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 108/217] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:20:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:20:59.874158 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3b59b-3a8f-4f5a-85b3-aa9cfb5479cb None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:59.899787 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6f10e-cce1-4a59-89d7-2e597741928c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:20:59.939883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc4916-55a6-4460-80ce-f279fcd9a438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:20:59.940688 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc4916-55a6-4460-80ce-f279fcd9a438 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:20:59.940972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc4916-55a6-4460-80ce-f279fcd9a438 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:20:59.941283 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc4916-55a6-4460-80ce-f279fcd9a438 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:20:59.952933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc4916-55a6-4460-80ce-f279fcd9a438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-60bc4916-55a6-4460-80ce-f279fcd9a438 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:20:59.961988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:20:59.962898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:20:59.963733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:20:59.963733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:20:59.963733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:20:59.963733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:20:59.963733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:20:59.963733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:20:59.963733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:20:59.963733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:20:59.963733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:20:59.963733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:20:59.963733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:20:59.963733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:20:59.963733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 110/218] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:20:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:20:59.965127 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc4916-55a6-4460-80ce-f279fcd9a438 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:20:59.985466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3b59b-3a8f-4f5a-85b3-aa9cfb5479cb None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:00.029709 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c29b61-355e-4dbb-9f23-b92f315aaef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:00.030487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c29b61-355e-4dbb-9f23-b92f315aaef5 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:00.030692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c29b61-355e-4dbb-9f23-b92f315aaef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:00.030950 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c29b61-355e-4dbb-9f23-b92f315aaef5 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:00.031891 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c29b61-355e-4dbb-9f23-b92f315aaef5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:00.033175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c29b61-355e-4dbb-9f23-b92f315aaef5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:00.040642 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:00.073363 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc4916-55a6-4460-80ce-f279fcd9a438 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:00.101792 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:00.107386 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660a468-9e45-48a8-85dc-773d50d43e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:00.108297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660a468-9e45-48a8-85dc-773d50d43e4d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:00.113331 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660a468-9e45-48a8-85dc-773d50d43e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:00.113331 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660a468-9e45-48a8-85dc-773d50d43e4d None admin] PATH_INFO: `/v3/users/60a70ee64c114c609ad233fdbd782cc2` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:00.122844 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660a468-9e45-48a8-85dc-773d50d43e4d None admin] RBAC: Authorizing `identity:get_user(user_id=60a70ee64c114c609ad233fdbd782cc2)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:00.124305 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660a468-9e45-48a8-85dc-773d50d43e4d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:00.126032 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 111/219] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:20:59 2022] GET /identity/v3/users/60a70ee64c114c609ad233fdbd782cc2 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:00.130681 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660a468-9e45-48a8-85dc-773d50d43e4d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:00.137951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 109/220] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:20:59 2022] POST /identity/v3/users => generated 297 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:21:00.137951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c29b61-355e-4dbb-9f23-b92f315aaef5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:00.256105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660a468-9e45-48a8-85dc-773d50d43e4d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:00.268559 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c29b61-355e-4dbb-9f23-b92f315aaef5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:00.296301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a7f66-aa9b-45d1-9d2b-cf841796f29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:00.297000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7f66-aa9b-45d1-9d2b-cf841796f29a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:00.297324 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7f66-aa9b-45d1-9d2b-cf841796f29a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:00.297604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7f66-aa9b-45d1-9d2b-cf841796f29a None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:00.298359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a7f66-aa9b-45d1-9d2b-cf841796f29a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:00.298664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a7f66-aa9b-45d1-9d2b-cf841796f29a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:00.300055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a7f66-aa9b-45d1-9d2b-cf841796f29a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:00.306647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7152150-cf7c-4262-9e6c-ada697e1ae04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:00.307324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7152150-cf7c-4262-9e6c-ada697e1ae04 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:00.307605 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7152150-cf7c-4262-9e6c-ada697e1ae04 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:00.307880 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7152150-cf7c-4262-9e6c-ada697e1ae04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:00.310909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 112/221] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:00.317031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d7ad0-ac9f-4222-9884-9c8dfbe3c8c2 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:00.317353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d7ad0-ac9f-4222-9884-9c8dfbe3c8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:00.317640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d7ad0-ac9f-4222-9884-9c8dfbe3c8c2 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:00.319227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 113/222] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:00.320497 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d7ad0-ac9f-4222-9884-9c8dfbe3c8c2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:00.381406 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:00.386425 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:00.392416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7152150-cf7c-4262-9e6c-ada697e1ae04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:00.393940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7152150-cf7c-4262-9e6c-ada697e1ae04 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:00.452244 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d7ad0-ac9f-4222-9884-9c8dfbe3c8c2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:00.492198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b63610-aed0-44e0-8dcb-a9534f26e40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:00.493111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b63610-aed0-44e0-8dcb-a9534f26e40f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:00.493408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b63610-aed0-44e0-8dcb-a9534f26e40f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:00.493693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b63610-aed0-44e0-8dcb-a9534f26e40f None admin] PATH_INFO: `/v3/users/9447d090257c48fbb6343720d2916622` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:00.494925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b63610-aed0-44e0-8dcb-a9534f26e40f None admin] RBAC: Authorizing `identity:get_user(user_id=9447d090257c48fbb6343720d2916622)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:00.496391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b63610-aed0-44e0-8dcb-a9534f26e40f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:00.498575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 114/223] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:21:00 2022] GET /identity/v3/users/9447d090257c48fbb6343720d2916622 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:00.499923 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c46b2-0648-4a43-a5e6-2cd9805824a3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:00.500248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c46b2-0648-4a43-a5e6-2cd9805824a3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:00.500504 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c46b2-0648-4a43-a5e6-2cd9805824a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:00.518350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-dc0c46b2-0648-4a43-a5e6-2cd9805824a3 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:00.519582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dc0c46b2-0648-4a43-a5e6-2cd9805824a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:00.520930 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 110/224] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:00.522670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7152150-cf7c-4262-9e6c-ada697e1ae04 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:00.593284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 115/225] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:00.594239 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c46b2-0648-4a43-a5e6-2cd9805824a3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:00.638635 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:00.648217 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7152150-cf7c-4262-9e6c-ada697e1ae04 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:00.725906 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:00.732031 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17265db-7ab5-4940-9e61-4d7a6f9fe974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:00.733499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17265db-7ab5-4940-9e61-4d7a6f9fe974 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:00.733793 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17265db-7ab5-4940-9e61-4d7a6f9fe974 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:00.734073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17265db-7ab5-4940-9e61-4d7a6f9fe974 None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:00.735425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17265db-7ab5-4940-9e61-4d7a6f9fe974 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:00.737120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17265db-7ab5-4940-9e61-4d7a6f9fe974 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:00.740045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 111/226] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:00 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:00.743094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17265db-7ab5-4940-9e61-4d7a6f9fe974 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:00.775387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c46b2-0648-4a43-a5e6-2cd9805824a3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:00.822727 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08735e-06f6-4412-92ac-a9f51250364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:00.824060 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08735e-06f6-4412-92ac-a9f51250364b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:00.824606 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08735e-06f6-4412-92ac-a9f51250364b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:00.825209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08735e-06f6-4412-92ac-a9f51250364b None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:00.826781 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08735e-06f6-4412-92ac-a9f51250364b None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:00.828425 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08735e-06f6-4412-92ac-a9f51250364b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:00.830626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 116/227] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:00 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:00.831591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08735e-06f6-4412-92ac-a9f51250364b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:00.866784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17265db-7ab5-4940-9e61-4d7a6f9fe974 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:00.903041 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a927f02-a4f7-4f0c-8f22-b5a4e5fd80a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:00.904214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a927f02-a4f7-4f0c-8f22-b5a4e5fd80a5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:00.904735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a927f02-a4f7-4f0c-8f22-b5a4e5fd80a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:00.905281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a927f02-a4f7-4f0c-8f22-b5a4e5fd80a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:00.916243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a927f02-a4f7-4f0c-8f22-b5a4e5fd80a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-7a927f02-a4f7-4f0c-8f22-b5a4e5fd80a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:00.924895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:00.925819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:00.926613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:00.926613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:00.926613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:00.926613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:00.926613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:00.926613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:00.926613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:00.926613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:00.926613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:00.926613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:00.926613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:00.926613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:00.930330 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 112/228] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:21:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:00.931488 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a927f02-a4f7-4f0c-8f22-b5a4e5fd80a5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:00.953685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:00.965252 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08735e-06f6-4412-92ac-a9f51250364b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:00.986151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:01.005769 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0854ac9-26da-439a-9dec-338cd84dd0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:01.007196 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0854ac9-26da-439a-9dec-338cd84dd0de None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:01.007701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0854ac9-26da-439a-9dec-338cd84dd0de None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:01.008277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0854ac9-26da-439a-9dec-338cd84dd0de None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10/users/9447d090257c48fbb6343720d2916622/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:01.010758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0854ac9-26da-439a-9dec-338cd84dd0de None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8a10b282fb412fbc79062901f84a10, user_id=9447d090257c48fbb6343720d2916622, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:01.012472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0854ac9-26da-439a-9dec-338cd84dd0de None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:01.053088 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a927f02-a4f7-4f0c-8f22-b5a4e5fd80a5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:01.077218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 117/229] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:21:00 2022] PUT /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10/users/9447d090257c48fbb6343720d2916622/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:21:01.078900 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0854ac9-26da-439a-9dec-338cd84dd0de None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:01.093019 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b65f6c-1a06-4a2d-849c-f897ac3c819a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:01.093723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b65f6c-1a06-4a2d-849c-f897ac3c819a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:01.094012 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b65f6c-1a06-4a2d-849c-f897ac3c819a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:01.094297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b65f6c-1a06-4a2d-849c-f897ac3c819a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:01.095146 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b65f6c-1a06-4a2d-849c-f897ac3c819a None admin] RBAC: Adding query filter params (role.id=9330473f62ec4a82a82d1c4beb6160d7, scope.project.id=bc8a10b282fb412fbc79062901f84a10, user.id=60a70ee64c114c609ad233fdbd782cc2) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:01.095465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b65f6c-1a06-4a2d-849c-f897ac3c819a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:01.097187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b65f6c-1a06-4a2d-849c-f897ac3c819a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:01.104915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 113/230] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:21:00 2022] GET /identity/v3/role_assignments?user.id=60a70ee64c114c609ad233fdbd782cc2&scope.project.id=bc8a10b282fb412fbc79062901f84a10&role.id=9330473f62ec4a82a82d1c4beb6160d7 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:01.202637 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0854ac9-26da-439a-9dec-338cd84dd0de None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:01.243681 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9604f5f-3a2a-4dd4-9bb7-6c4099744175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:01.244774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9604f5f-3a2a-4dd4-9bb7-6c4099744175 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:01.245326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9604f5f-3a2a-4dd4-9bb7-6c4099744175 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:01.245897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9604f5f-3a2a-4dd4-9bb7-6c4099744175 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:01.246867 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9604f5f-3a2a-4dd4-9bb7-6c4099744175 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:01.247376 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9604f5f-3a2a-4dd4-9bb7-6c4099744175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:01.248971 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9604f5f-3a2a-4dd4-9bb7-6c4099744175 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:01.260050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 118/231] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:01.265604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b65f6c-1a06-4a2d-849c-f897ac3c819a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:01.393180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:01.416107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b65f6c-1a06-4a2d-849c-f897ac3c819a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:01.459051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398fefa-84db-43c1-9210-19e61971bc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:01.460276 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398fefa-84db-43c1-9210-19e61971bc70 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:01.460714 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398fefa-84db-43c1-9210-19e61971bc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:01.461006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398fefa-84db-43c1-9210-19e61971bc70 None admin] PATH_INFO: `/v3/users/9447d090257c48fbb6343720d2916622` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:01.462294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398fefa-84db-43c1-9210-19e61971bc70 None admin] RBAC: Authorizing `identity:get_user(user_id=9447d090257c48fbb6343720d2916622)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:01.463751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398fefa-84db-43c1-9210-19e61971bc70 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:01.465758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 114/232] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:21:01 2022] GET /identity/v3/users/9447d090257c48fbb6343720d2916622 => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:01.471251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9604f5f-3a2a-4dd4-9bb7-6c4099744175 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:01.521214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:01.596683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9604f5f-3a2a-4dd4-9bb7-6c4099744175 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:01.638003 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fdd69-7710-48b7-a9b8-c53278606193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:01.639360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fdd69-7710-48b7-a9b8-c53278606193 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:01.639861 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fdd69-7710-48b7-a9b8-c53278606193 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:01.640392 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fdd69-7710-48b7-a9b8-c53278606193 None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:01.645872 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fdd69-7710-48b7-a9b8-c53278606193 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:01.648128 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fdd69-7710-48b7-a9b8-c53278606193 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:01.650460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 119/233] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:01 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:01.656073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398fefa-84db-43c1-9210-19e61971bc70 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:01.814920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398fefa-84db-43c1-9210-19e61971bc70 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:01.858892 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf060f2-e5ed-4d9d-80d1-44e8d2ada2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:01.859804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf060f2-e5ed-4d9d-80d1-44e8d2ada2b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:01.860095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf060f2-e5ed-4d9d-80d1-44e8d2ada2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:01.860381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf060f2-e5ed-4d9d-80d1-44e8d2ada2b5 None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10/users/9447d090257c48fbb6343720d2916622/roles/1bac0428b2544e2cb16fad539f080c14` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:01.862688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf060f2-e5ed-4d9d-80d1-44e8d2ada2b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8a10b282fb412fbc79062901f84a10, user_id=9447d090257c48fbb6343720d2916622, role_id=1bac0428b2544e2cb16fad539f080c14)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:01.864170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf060f2-e5ed-4d9d-80d1-44e8d2ada2b5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:01.960555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 115/234] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:21:01 2022] PUT /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10/users/9447d090257c48fbb6343720d2916622/roles/1bac0428b2544e2cb16fad539f080c14 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:21:02.704998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82867136-6e7a-4774-8890-e01f3839a06f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:02.705860 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82867136-6e7a-4774-8890-e01f3839a06f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:02.706336 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82867136-6e7a-4774-8890-e01f3839a06f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:02.708184 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 120/235] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:02.713610 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324a171-8e2d-45b9-a78c-5356ccfae159 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:02.714034 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324a171-8e2d-45b9-a78c-5356ccfae159 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:02.714187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324a171-8e2d-45b9-a78c-5356ccfae159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:02.743153 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b324a171-8e2d-45b9-a78c-5356ccfae159 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:02.744589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b324a171-8e2d-45b9-a78c-5356ccfae159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:02.750095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:02.836051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 116/236] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:02.871043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82867136-6e7a-4774-8890-e01f3839a06f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:03.005849 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82867136-6e7a-4774-8890-e01f3839a06f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:03.013660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:03.051129 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad519d6a-b272-4e2e-8a3d-5122bb6a2872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:03.051955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad519d6a-b272-4e2e-8a3d-5122bb6a2872 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:03.052243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad519d6a-b272-4e2e-8a3d-5122bb6a2872 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:03.052524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad519d6a-b272-4e2e-8a3d-5122bb6a2872 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:03.075759 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad519d6a-b272-4e2e-8a3d-5122bb6a2872 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-ad519d6a-b272-4e2e-8a3d-5122bb6a2872 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:03.088118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:03.089130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:03.089951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:03.089951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:03.089951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:03.089951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:03.089951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:03.089951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:03.089951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:03.089951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:03.089951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:03.089951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:03.089951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:21:03.089951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:03.092504 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 121/237] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:03.098469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324a171-8e2d-45b9-a78c-5356ccfae159 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:03.229976 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324a171-8e2d-45b9-a78c-5356ccfae159 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:03.284987 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214843b-e737-4ef3-be0b-3130c13ccd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:03.285705 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214843b-e737-4ef3-be0b-3130c13ccd9d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:03.285998 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214843b-e737-4ef3-be0b-3130c13ccd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:03.286281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214843b-e737-4ef3-be0b-3130c13ccd9d None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:03.287045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214843b-e737-4ef3-be0b-3130c13ccd9d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:03.287367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214843b-e737-4ef3-be0b-3130c13ccd9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:03.288830 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214843b-e737-4ef3-be0b-3130c13ccd9d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:03.296161 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0704a-7295-4f12-bf1c-b2f77149ed26 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:03.296456 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0704a-7295-4f12-bf1c-b2f77149ed26 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:03.296738 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0704a-7295-4f12-bf1c-b2f77149ed26 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:03.298405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 122/238] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:03.301005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 117/239] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:03.307417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3fd7c-4b6f-4266-9a3e-d1dbf9c39d94 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:03.307709 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3fd7c-4b6f-4266-9a3e-d1dbf9c39d94 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:03.307993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3fd7c-4b6f-4266-9a3e-d1dbf9c39d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:03.311316 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214843b-e737-4ef3-be0b-3130c13ccd9d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:03.316485 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:03.337092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f9a3fd7c-4b6f-4266-9a3e-d1dbf9c39d94 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:03.338306 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f9a3fd7c-4b6f-4266-9a3e-d1dbf9c39d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:03.466269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214843b-e737-4ef3-be0b-3130c13ccd9d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:03.467606 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 123/240] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:03.511288 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3fd7c-4b6f-4266-9a3e-d1dbf9c39d94 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:03.524151 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e26ae4-a5fc-4933-886d-090be68d06f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:03.525130 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e26ae4-a5fc-4933-886d-090be68d06f0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:03.525528 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e26ae4-a5fc-4933-886d-090be68d06f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:03.525814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e26ae4-a5fc-4933-886d-090be68d06f0 None admin] PATH_INFO: `/v3/users/2d2917634d1846bcaed69120a3386c81` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:03.541980 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e26ae4-a5fc-4933-886d-090be68d06f0 None admin] RBAC: Authorizing `identity:get_user(user_id=2d2917634d1846bcaed69120a3386c81)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:03.543672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e26ae4-a5fc-4933-886d-090be68d06f0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:03.545565 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 118/241] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:21:03 2022] GET /identity/v3/users/2d2917634d1846bcaed69120a3386c81 => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:21:03.551209 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e26ae4-a5fc-4933-886d-090be68d06f0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:03.659110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:03.703775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:03.709640 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e26ae4-a5fc-4933-886d-090be68d06f0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:03.718014 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3fd7c-4b6f-4266-9a3e-d1dbf9c39d94 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:03.781083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccde887-9949-4066-aca2-6eac9159c5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:03.782931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccde887-9949-4066-aca2-6eac9159c5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:03.783906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccde887-9949-4066-aca2-6eac9159c5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:03.785682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccde887-9949-4066-aca2-6eac9159c5a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:03.787074 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ff905-241b-4d66-83c4-70c08329b33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:03.788758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ff905-241b-4d66-83c4-70c08329b33f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:03.789276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ff905-241b-4d66-83c4-70c08329b33f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:03.789720 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ff905-241b-4d66-83c4-70c08329b33f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:03.791097 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ff905-241b-4d66-83c4-70c08329b33f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:03.792781 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ff905-241b-4d66-83c4-70c08329b33f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:03.797848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 124/242] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:21:03.797848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bb44c-0516-4887-83ec-36f5d3190ba3 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:03.797848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bb44c-0516-4887-83ec-36f5d3190ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:03.797848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bb44c-0516-4887-83ec-36f5d3190ba3 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:03.797848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 125/243] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:03.802276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b49a92-6f5f-46ba-832f-0e764bf51bbe None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:03.802568 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b49a92-6f5f-46ba-832f-0e764bf51bbe None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:03.802848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b49a92-6f5f-46ba-832f-0e764bf51bbe None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:03.804887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 126/244] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:03.805496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b49a92-6f5f-46ba-832f-0e764bf51bbe None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:03.868057 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccde887-9949-4066-aca2-6eac9159c5a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:03.869595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccde887-9949-4066-aca2-6eac9159c5a7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:03.917835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b49a92-6f5f-46ba-832f-0e764bf51bbe None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:03.925538 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:03.955430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1905e-d099-421c-b48a-de7f2e407ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:03.956378 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1905e-d099-421c-b48a-de7f2e407ab2 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:03.956664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1905e-d099-421c-b48a-de7f2e407ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:03.956942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1905e-d099-421c-b48a-de7f2e407ab2 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:03.957837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1905e-d099-421c-b48a-de7f2e407ab2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:03.960675 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1905e-d099-421c-b48a-de7f2e407ab2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:03.971848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 119/245] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:21:03.971848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49846538-ae45-480a-9abc-2cbdc5d43ad2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:03.971848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49846538-ae45-480a-9abc-2cbdc5d43ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:03.971848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49846538-ae45-480a-9abc-2cbdc5d43ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:03.985789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-49846538-ae45-480a-9abc-2cbdc5d43ad2 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:03.986965 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-49846538-ae45-480a-9abc-2cbdc5d43ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:04.050883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 127/246] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:21:03 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:21:04.052095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519e1f7-2c2d-4d33-92f5-ad2268ef23d6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:04.052391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519e1f7-2c2d-4d33-92f5-ad2268ef23d6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:04.052841 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519e1f7-2c2d-4d33-92f5-ad2268ef23d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:04.069533 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 120/247] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:04.072247 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49846538-ae45-480a-9abc-2cbdc5d43ad2 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:04.079279 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d519e1f7-2c2d-4d33-92f5-ad2268ef23d6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:04.085752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d519e1f7-2c2d-4d33-92f5-ad2268ef23d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:04.104169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:04.182871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 128/248] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:04.184117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519e1f7-2c2d-4d33-92f5-ad2268ef23d6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:04.235329 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:04.255873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49846538-ae45-480a-9abc-2cbdc5d43ad2 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:04.300411 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c2d2a-a8ed-412d-924f-33156d923820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:04.301793 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c2d2a-a8ed-412d-924f-33156d923820 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:04.301793 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c2d2a-a8ed-412d-924f-33156d923820 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:04.301980 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c2d2a-a8ed-412d-924f-33156d923820 None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:04.303780 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c2d2a-a8ed-412d-924f-33156d923820 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:04.307127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c2d2a-a8ed-412d-924f-33156d923820 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:04.310076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 121/249] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:04 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:04.310957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c2d2a-a8ed-412d-924f-33156d923820 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:04.317360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519e1f7-2c2d-4d33-92f5-ad2268ef23d6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:04.365044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece13425-f95f-428c-8406-c93450a3722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:04.365044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece13425-f95f-428c-8406-c93450a3722c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:04.365044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece13425-f95f-428c-8406-c93450a3722c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:04.365044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece13425-f95f-428c-8406-c93450a3722c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:04.380008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece13425-f95f-428c-8406-c93450a3722c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-ece13425-f95f-428c-8406-c93450a3722c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:04.389998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:04.390957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:04.391773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:04.391773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:04.391773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:04.391773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:04.391773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:04.391773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:04.391773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:04.391773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:04.391773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:04.391773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:04.391773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:04.391773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:04.391773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 129/250] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:04.392442 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece13425-f95f-428c-8406-c93450a3722c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:04.408647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:04.450562 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c2d2a-a8ed-412d-924f-33156d923820 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:04.490354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ad744-83bf-46de-af49-5f70c1350055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:04.491756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ad744-83bf-46de-af49-5f70c1350055 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:04.492140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ad744-83bf-46de-af49-5f70c1350055 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:04.492430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ad744-83bf-46de-af49-5f70c1350055 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:04.505902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ad744-83bf-46de-af49-5f70c1350055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:04.514004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece13425-f95f-428c-8406-c93450a3722c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-386ad744-83bf-46de-af49-5f70c1350055 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:04.514537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:04.515441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:04.516272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:04.516272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:04.516272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:04.516272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:04.516272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:04.516272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:04.516272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:04.516272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:04.516272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:04.516272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:04.516272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:21:04.516272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:04.516272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 122/251] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:21:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:04.517650 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ad744-83bf-46de-af49-5f70c1350055 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:04.540185 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:04.565101 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7854e-1f1a-4cab-875e-3700dbe736d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:04.565701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7854e-1f1a-4cab-875e-3700dbe736d2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:04.565995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7854e-1f1a-4cab-875e-3700dbe736d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:04.566280 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7854e-1f1a-4cab-875e-3700dbe736d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:04.567522 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7854e-1f1a-4cab-875e-3700dbe736d2 None admin] RBAC: Adding query filter params (role.id=1026857a0bd64410a463448b53532814, scope.project.id=bc8a10b282fb412fbc79062901f84a10, user.id=2d2917634d1846bcaed69120a3386c81) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:04.567845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7854e-1f1a-4cab-875e-3700dbe736d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:04.570715 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7854e-1f1a-4cab-875e-3700dbe736d2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:04.578941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 130/252] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:21:04 2022] GET /identity/v3/role_assignments?user.id=2d2917634d1846bcaed69120a3386c81&scope.project.id=bc8a10b282fb412fbc79062901f84a10&role.id=1026857a0bd64410a463448b53532814 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:04.579836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7854e-1f1a-4cab-875e-3700dbe736d2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:04.727643 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ad744-83bf-46de-af49-5f70c1350055 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:04.740003 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:04.786652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7854e-1f1a-4cab-875e-3700dbe736d2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:04.830253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da6fbb-a3f6-490f-8236-fb534a2b6107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:04.832105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da6fbb-a3f6-490f-8236-fb534a2b6107 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:04.832910 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbcfd0-4918-4428-bbd7-e4ec0caab355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:04.833773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbcfd0-4918-4428-bbd7-e4ec0caab355 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:04.834763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da6fbb-a3f6-490f-8236-fb534a2b6107 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:04.835284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbcfd0-4918-4428-bbd7-e4ec0caab355 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:04.835710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbcfd0-4918-4428-bbd7-e4ec0caab355 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:04.836153 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da6fbb-a3f6-490f-8236-fb534a2b6107 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:04.837087 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbcfd0-4918-4428-bbd7-e4ec0caab355 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:04.838026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da6fbb-a3f6-490f-8236-fb534a2b6107 None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:04.838504 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbcfd0-4918-4428-bbd7-e4ec0caab355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:04.838967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da6fbb-a3f6-490f-8236-fb534a2b6107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:04.840563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbcfd0-4918-4428-bbd7-e4ec0caab355 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:04.842206 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da6fbb-a3f6-490f-8236-fb534a2b6107 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:04.860196 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 131/253] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:04.862838 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 123/254] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:04.872311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbcfd0-4918-4428-bbd7-e4ec0caab355 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:04.873475 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da6fbb-a3f6-490f-8236-fb534a2b6107 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:05.002548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:05.015325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da6fbb-a3f6-490f-8236-fb534a2b6107 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:05.026979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbcfd0-4918-4428-bbd7-e4ec0caab355 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:05.051804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4c966-5bbf-4307-82dc-d3f4231dac0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:05.052692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4c966-5bbf-4307-82dc-d3f4231dac0e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:05.053050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4c966-5bbf-4307-82dc-d3f4231dac0e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:05.054100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4c966-5bbf-4307-82dc-d3f4231dac0e None admin] PATH_INFO: `/v3/users/9447d090257c48fbb6343720d2916622` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:05.055511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4c966-5bbf-4307-82dc-d3f4231dac0e None admin] RBAC: Authorizing `identity:get_user(user_id=9447d090257c48fbb6343720d2916622)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:05.057139 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4c966-5bbf-4307-82dc-d3f4231dac0e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:05.058933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 124/255] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:21:04 2022] GET /identity/v3/users/9447d090257c48fbb6343720d2916622 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:05.060482 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55c5ec-5b7d-44b8-9f31-019a6b3f239b None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:05.060920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55c5ec-5b7d-44b8-9f31-019a6b3f239b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:05.061360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55c5ec-5b7d-44b8-9f31-019a6b3f239b None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:05.063006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 125/256] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:05.064301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55c5ec-5b7d-44b8-9f31-019a6b3f239b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:05.097903 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be381a-75c1-4fa2-8e65-2beb8e483e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:05.098769 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be381a-75c1-4fa2-8e65-2beb8e483e06 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:05.099111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be381a-75c1-4fa2-8e65-2beb8e483e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:05.099389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be381a-75c1-4fa2-8e65-2beb8e483e06 None admin] PATH_INFO: `/v3/users/60a70ee64c114c609ad233fdbd782cc2` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:05.103612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be381a-75c1-4fa2-8e65-2beb8e483e06 None admin] RBAC: Authorizing `identity:get_user(user_id=60a70ee64c114c609ad233fdbd782cc2)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:05.105380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be381a-75c1-4fa2-8e65-2beb8e483e06 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:05.111512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 132/257] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:21:04 2022] GET /identity/v3/users/60a70ee64c114c609ad233fdbd782cc2 => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:05.111512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0cce1-fefe-4110-9fee-8b8cba801af7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:05.111512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0cce1-fefe-4110-9fee-8b8cba801af7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:05.111512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0cce1-fefe-4110-9fee-8b8cba801af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:05.128661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:05.150473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-eba0cce1-fefe-4110-9fee-8b8cba801af7 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:05.152831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-eba0cce1-fefe-4110-9fee-8b8cba801af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:05.234748 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 133/258] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:05.235993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0cce1-fefe-4110-9fee-8b8cba801af7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:05.267956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55c5ec-5b7d-44b8-9f31-019a6b3f239b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:05.327398 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045482d-0dc3-4fd7-9926-94b1bfa305a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:05.328315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045482d-0dc3-4fd7-9926-94b1bfa305a4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:05.328779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045482d-0dc3-4fd7-9926-94b1bfa305a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:05.329262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045482d-0dc3-4fd7-9926-94b1bfa305a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:05.349644 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:05.391984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0cce1-fefe-4110-9fee-8b8cba801af7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:05.404092 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045482d-0dc3-4fd7-9926-94b1bfa305a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:05.405970 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045482d-0dc3-4fd7-9926-94b1bfa305a4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:05.460808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa37482-0ce7-450f-8b00-4d2c80cff1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:05.462066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa37482-0ce7-450f-8b00-4d2c80cff1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:05.462581 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa37482-0ce7-450f-8b00-4d2c80cff1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:05.463060 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa37482-0ce7-450f-8b00-4d2c80cff1f1 None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:05.465976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa37482-0ce7-450f-8b00-4d2c80cff1f1 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:05.471983 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa37482-0ce7-450f-8b00-4d2c80cff1f1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:05.473981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 134/259] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:05 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:05.475237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa37482-0ce7-450f-8b00-4d2c80cff1f1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:05.481363 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:05.565800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 126/260] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:21:05.566810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045482d-0dc3-4fd7-9926-94b1bfa305a4 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:05.628439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:05.686776 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa37482-0ce7-450f-8b00-4d2c80cff1f1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:05.748294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045482d-0dc3-4fd7-9926-94b1bfa305a4 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:05.770056 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafaea1-c710-4ebc-8096-6c03ff5635df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:05.770879 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafaea1-c710-4ebc-8096-6c03ff5635df None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:05.771166 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafaea1-c710-4ebc-8096-6c03ff5635df None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:05.771443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafaea1-c710-4ebc-8096-6c03ff5635df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:05.790920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafaea1-c710-4ebc-8096-6c03ff5635df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-bfafaea1-c710-4ebc-8096-6c03ff5635df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:05.799764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:05.800669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:05.801519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:05.801519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:05.801519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:05.801519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:05.801519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:05.801519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:05.801519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:05.801519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:05.801519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:05.801519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:05.801519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:05.801519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:05.805402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 135/261] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:21:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:05.807066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafaea1-c710-4ebc-8096-6c03ff5635df None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:05.848939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390fcba-655c-4706-9837-40a1b0921649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:05.851322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390fcba-655c-4706-9837-40a1b0921649 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:05.851765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390fcba-655c-4706-9837-40a1b0921649 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:05.852186 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390fcba-655c-4706-9837-40a1b0921649 None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10/users/60a70ee64c114c609ad233fdbd782cc2/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:05.854423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390fcba-655c-4706-9837-40a1b0921649 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8a10b282fb412fbc79062901f84a10, user_id=60a70ee64c114c609ad233fdbd782cc2, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:05.865082 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390fcba-655c-4706-9837-40a1b0921649 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:05.875833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:05.915967 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafaea1-c710-4ebc-8096-6c03ff5635df None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:05.951830 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daee6d3-76ef-4356-b42a-5dcb8263855c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:05.953491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daee6d3-76ef-4356-b42a-5dcb8263855c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:05.954646 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daee6d3-76ef-4356-b42a-5dcb8263855c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:05.954936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daee6d3-76ef-4356-b42a-5dcb8263855c None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:05.956344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daee6d3-76ef-4356-b42a-5dcb8263855c None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:05.958513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daee6d3-76ef-4356-b42a-5dcb8263855c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:05.961255 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 127/262] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:21:05 2022] PUT /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10/users/60a70ee64c114c609ad233fdbd782cc2/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:21:05.965921 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390fcba-655c-4706-9837-40a1b0921649 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:05.969395 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 136/263] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:05 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:05.975753 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daee6d3-76ef-4356-b42a-5dcb8263855c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:06.211878 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390fcba-655c-4706-9837-40a1b0921649 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:06.222394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daee6d3-76ef-4356-b42a-5dcb8263855c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:06.232129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:06.265899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923f6ba-0425-48b1-8bd1-7fcd49cdcfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:06.266598 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923f6ba-0425-48b1-8bd1-7fcd49cdcfa4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:06.266925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923f6ba-0425-48b1-8bd1-7fcd49cdcfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:06.267210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923f6ba-0425-48b1-8bd1-7fcd49cdcfa4 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:06.269391 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923f6ba-0425-48b1-8bd1-7fcd49cdcfa4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:06.269735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923f6ba-0425-48b1-8bd1-7fcd49cdcfa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:06.271471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923f6ba-0425-48b1-8bd1-7fcd49cdcfa4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:06.282501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 128/264] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:06.291371 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923f6ba-0425-48b1-8bd1-7fcd49cdcfa4 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:06.294443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:06.310558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cd492-1afb-415d-8338-cfd5f5db2ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:06.311234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cd492-1afb-415d-8338-cfd5f5db2ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:06.311527 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cd492-1afb-415d-8338-cfd5f5db2ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:06.311813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cd492-1afb-415d-8338-cfd5f5db2ad5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:06.312698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cd492-1afb-415d-8338-cfd5f5db2ad5 None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=bc8a10b282fb412fbc79062901f84a10, user.id=9447d090257c48fbb6343720d2916622) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:06.313020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cd492-1afb-415d-8338-cfd5f5db2ad5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:06.314517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cd492-1afb-415d-8338-cfd5f5db2ad5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:06.324592 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 137/265] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:21:05 2022] GET /identity/v3/role_assignments?user.id=9447d090257c48fbb6343720d2916622&scope.project.id=bc8a10b282fb412fbc79062901f84a10&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 608 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:06.450364 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923f6ba-0425-48b1-8bd1-7fcd49cdcfa4 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:06.491633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6cf39-a62b-4724-a8a6-fa26a5df303f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:06.492451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6cf39-a62b-4724-a8a6-fa26a5df303f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:06.492783 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6cf39-a62b-4724-a8a6-fa26a5df303f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:06.493026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6cf39-a62b-4724-a8a6-fa26a5df303f None admin] PATH_INFO: `/v3/users/9447d090257c48fbb6343720d2916622` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:06.494320 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6cf39-a62b-4724-a8a6-fa26a5df303f None admin] RBAC: Authorizing `identity:get_user(user_id=9447d090257c48fbb6343720d2916622)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:06.495806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6cf39-a62b-4724-a8a6-fa26a5df303f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:06.497804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 129/266] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:21:06 2022] GET /identity/v3/users/9447d090257c48fbb6343720d2916622 => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:06.503327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cd492-1afb-415d-8338-cfd5f5db2ad5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:06.618246 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cd492-1afb-415d-8338-cfd5f5db2ad5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:06.660834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a71b0-b534-47f9-909c-cacb41f24a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:06.661550 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a71b0-b534-47f9-909c-cacb41f24a98 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:06.661846 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a71b0-b534-47f9-909c-cacb41f24a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:06.662130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a71b0-b534-47f9-909c-cacb41f24a98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:06.762345 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:06.770418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a71b0-b534-47f9-909c-cacb41f24a98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:06.772451 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a71b0-b534-47f9-909c-cacb41f24a98 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:06.911833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 138/267] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:06.918083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6cf39-a62b-4724-a8a6-fa26a5df303f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:06.963681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:07.040164 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6cf39-a62b-4724-a8a6-fa26a5df303f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:07.080127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af89d97-b494-48d9-b162-57422549524f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:07.081028 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af89d97-b494-48d9-b162-57422549524f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:07.081367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af89d97-b494-48d9-b162-57422549524f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:07.081652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af89d97-b494-48d9-b162-57422549524f None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:07.082874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af89d97-b494-48d9-b162-57422549524f None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:07.084635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a0752-c0e3-45ce-8c33-cd20686010c6 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:07.084907 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af89d97-b494-48d9-b162-57422549524f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:07.085163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a0752-c0e3-45ce-8c33-cd20686010c6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:07.085377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a0752-c0e3-45ce-8c33-cd20686010c6 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:07.087039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 130/269] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:06 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:07.087158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 139/269] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:07.091776 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39f55a-a913-4a11-95f5-63f0a80bce7f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:07.092073 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39f55a-a913-4a11-95f5-63f0a80bce7f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:07.092360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39f55a-a913-4a11-95f5-63f0a80bce7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:07.096402 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af89d97-b494-48d9-b162-57422549524f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:07.124976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3c39f55a-a913-4a11-95f5-63f0a80bce7f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:07.129872 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3c39f55a-a913-4a11-95f5-63f0a80bce7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:07.265997 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 140/270] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:07.272652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af89d97-b494-48d9-b162-57422549524f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:07.280192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:07.307155 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39f55a-a913-4a11-95f5-63f0a80bce7f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:07.332325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beac80a-bc45-4b84-bbf5-adbf25153d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:07.333000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beac80a-bc45-4b84-bbf5-adbf25153d42 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:07.333322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beac80a-bc45-4b84-bbf5-adbf25153d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:07.333612 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beac80a-bc45-4b84-bbf5-adbf25153d42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:07.334584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beac80a-bc45-4b84-bbf5-adbf25153d42 None admin] RBAC: Adding query filter params (role.id=1bac0428b2544e2cb16fad539f080c14, scope.project.id=bc8a10b282fb412fbc79062901f84a10, user.id=9447d090257c48fbb6343720d2916622) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:07.334911 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beac80a-bc45-4b84-bbf5-adbf25153d42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:07.336372 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beac80a-bc45-4b84-bbf5-adbf25153d42 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:07.346415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:07.349532 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 131/271] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:21:07 2022] GET /identity/v3/role_assignments?user.id=9447d090257c48fbb6343720d2916622&scope.project.id=bc8a10b282fb412fbc79062901f84a10&role.id=1bac0428b2544e2cb16fad539f080c14 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:21:07.404926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f403f-b66c-489b-8486-4d0d3c16be40 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:07.405270 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f403f-b66c-489b-8486-4d0d3c16be40 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:07.405557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f403f-b66c-489b-8486-4d0d3c16be40 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:07.407176 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 132/272] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:21:07.413218 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4acdb-b6b4-49ac-8c9c-7b5c9e63f2d9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:07.413553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4acdb-b6b4-49ac-8c9c-7b5c9e63f2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:07.413779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4acdb-b6b4-49ac-8c9c-7b5c9e63f2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:07.440455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7de4acdb-b6b4-49ac-8c9c-7b5c9e63f2d9 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:07.442861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7de4acdb-b6b4-49ac-8c9c-7b5c9e63f2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:07.457151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39f55a-a913-4a11-95f5-63f0a80bce7f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:07.499587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be9cfc-73bf-4225-b76c-09530ad15e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:07.500536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be9cfc-73bf-4225-b76c-09530ad15e6e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:07.500900 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be9cfc-73bf-4225-b76c-09530ad15e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:07.501285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be9cfc-73bf-4225-b76c-09530ad15e6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:07.514974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be9cfc-73bf-4225-b76c-09530ad15e6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-e1be9cfc-73bf-4225-b76c-09530ad15e6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:07.524264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:07.525298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:07.526473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:07.526473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:07.526473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:07.526473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:07.526473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:07.526473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:07.526473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:07.526473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:07.526473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:07.526473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:07.526473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:07.526473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:07.526473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 141/273] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:07.532460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bffa9-6495-447f-bbb5-aaf4ffb60169 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:07.532820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bffa9-6495-447f-bbb5-aaf4ffb60169 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:07.533224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bffa9-6495-447f-bbb5-aaf4ffb60169 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:07.535067 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 142/274] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:07.535950 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bffa9-6495-447f-bbb5-aaf4ffb60169 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:07.544112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 133/275] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:21:07.545891 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390df541-b41c-4a1e-991c-12bd0a8507ed None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:07.546401 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390df541-b41c-4a1e-991c-12bd0a8507ed None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:07.549379 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390df541-b41c-4a1e-991c-12bd0a8507ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:07.587558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-390df541-b41c-4a1e-991c-12bd0a8507ed None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:07.588679 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-390df541-b41c-4a1e-991c-12bd0a8507ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:07.659773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:07.674345 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 134/276] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:07.675703 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390df541-b41c-4a1e-991c-12bd0a8507ed None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:07.687001 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bffa9-6495-447f-bbb5-aaf4ffb60169 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:07.727395 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:07.734316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e79623-7dc8-4c38-b558-8e1331b664c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:07.735005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e79623-7dc8-4c38-b558-8e1331b664c4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:07.735296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e79623-7dc8-4c38-b558-8e1331b664c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:07.735577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e79623-7dc8-4c38-b558-8e1331b664c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:07.736334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e79623-7dc8-4c38-b558-8e1331b664c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:07.736647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e79623-7dc8-4c38-b558-8e1331b664c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:07.738081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e79623-7dc8-4c38-b558-8e1331b664c4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:07.747836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 143/277] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:07.748677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e79623-7dc8-4c38-b558-8e1331b664c4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:07.804194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390df541-b41c-4a1e-991c-12bd0a8507ed None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:07.840730 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b70c24-1296-4cd7-97f7-5a5519cb0ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:07.842002 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b70c24-1296-4cd7-97f7-5a5519cb0ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:07.842474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b70c24-1296-4cd7-97f7-5a5519cb0ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:07.842933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b70c24-1296-4cd7-97f7-5a5519cb0ea7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:07.844276 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b70c24-1296-4cd7-97f7-5a5519cb0ea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:07.846050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b70c24-1296-4cd7-97f7-5a5519cb0ea7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:07.853050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 135/278] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:07.854258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b70c24-1296-4cd7-97f7-5a5519cb0ea7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:07.856090 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e79623-7dc8-4c38-b558-8e1331b664c4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:07.891328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52219149-9ea8-41f6-801f-8b2b4cb96baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:07.892130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52219149-9ea8-41f6-801f-8b2b4cb96baa None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:07.892415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52219149-9ea8-41f6-801f-8b2b4cb96baa None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:07.892691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52219149-9ea8-41f6-801f-8b2b4cb96baa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:07.903799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52219149-9ea8-41f6-801f-8b2b4cb96baa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-52219149-9ea8-41f6-801f-8b2b4cb96baa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:07.911595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:07.912562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:07.914228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:07.914228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:07.914228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:07.914228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:07.914228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:07.914228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:07.914228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:07.914228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:07.914228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:07.914228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:07.914228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:21:07.914228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:07.914228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 144/279] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:07.914228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52219149-9ea8-41f6-801f-8b2b4cb96baa None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:07.923738 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:07.961682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:07.966870 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b70c24-1296-4cd7-97f7-5a5519cb0ea7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:08.001949 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6696941-8b5f-4e9c-beb6-2bbd5af473a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:08.002900 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6696941-8b5f-4e9c-beb6-2bbd5af473a2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:08.003459 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6696941-8b5f-4e9c-beb6-2bbd5af473a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:08.003459 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6696941-8b5f-4e9c-beb6-2bbd5af473a2 None admin] PATH_INFO: `/v3/users/8fce566f37654115a97c394065c7d7fc` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:08.018646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6696941-8b5f-4e9c-beb6-2bbd5af473a2 None admin] RBAC: Authorizing `identity:get_user(user_id=8fce566f37654115a97c394065c7d7fc)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:08.020117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6696941-8b5f-4e9c-beb6-2bbd5af473a2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:08.022042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 136/280] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:21:07 2022] GET /identity/v3/users/8fce566f37654115a97c394065c7d7fc => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:08.022826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6696941-8b5f-4e9c-beb6-2bbd5af473a2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:08.028650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52219149-9ea8-41f6-801f-8b2b4cb96baa None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:08.064366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aaf661-9103-4954-9758-719a27bbf86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:08.065030 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aaf661-9103-4954-9758-719a27bbf86d None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:08.065363 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aaf661-9103-4954-9758-719a27bbf86d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:08.065687 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aaf661-9103-4954-9758-719a27bbf86d None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:08.066389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aaf661-9103-4954-9758-719a27bbf86d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:08.067836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aaf661-9103-4954-9758-719a27bbf86d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:08.123726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6696941-8b5f-4e9c-beb6-2bbd5af473a2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:08.130838 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 145/281] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:21:07 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:21:08.131951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aaf661-9103-4954-9758-719a27bbf86d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:08.179181 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccb7c7-32d0-4f6c-a070-ba23e5fc43e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:08.180187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccb7c7-32d0-4f6c-a070-ba23e5fc43e8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:08.180899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccb7c7-32d0-4f6c-a070-ba23e5fc43e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:08.181395 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccb7c7-32d0-4f6c-a070-ba23e5fc43e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:08.183477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccb7c7-32d0-4f6c-a070-ba23e5fc43e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:08.184020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccb7c7-32d0-4f6c-a070-ba23e5fc43e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:08.185636 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccb7c7-32d0-4f6c-a070-ba23e5fc43e8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:08.198634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 137/282] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:08.204596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccb7c7-32d0-4f6c-a070-ba23e5fc43e8 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:08.219662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:08.236065 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:08.270810 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aaf661-9103-4954-9758-719a27bbf86d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:08.310243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c662249-7eb4-44b7-b007-591b30907267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:08.311290 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c662249-7eb4-44b7-b007-591b30907267 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:08.311874 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c662249-7eb4-44b7-b007-591b30907267 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:08.312409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c662249-7eb4-44b7-b007-591b30907267 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:08.315056 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c662249-7eb4-44b7-b007-591b30907267 None admin] RBAC: Adding query filter params (role.id=9330473f62ec4a82a82d1c4beb6160d7, user.id=8fce566f37654115a97c394065c7d7fc, scope.system=all) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:08.315654 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c662249-7eb4-44b7-b007-591b30907267 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:08.317669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c662249-7eb4-44b7-b007-591b30907267 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:08.330871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 146/283] 10.222.0.64 () {56 vars in 1308 bytes} [Mon May 23 15:21:08 2022] GET /identity/v3/role_assignments?user.id=8fce566f37654115a97c394065c7d7fc&scope.system=all&role.id=9330473f62ec4a82a82d1c4beb6160d7 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:08.356851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccb7c7-32d0-4f6c-a070-ba23e5fc43e8 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:08.394863 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dbacc-3614-4c57-99a3-a76a4379515a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:08.396062 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dbacc-3614-4c57-99a3-a76a4379515a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:08.397133 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dbacc-3614-4c57-99a3-a76a4379515a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:08.397675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dbacc-3614-4c57-99a3-a76a4379515a None admin] PATH_INFO: `/v3/users/2d2917634d1846bcaed69120a3386c81` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:08.399446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038dbacc-3614-4c57-99a3-a76a4379515a None admin] RBAC: Authorizing `identity:get_user(user_id=2d2917634d1846bcaed69120a3386c81)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:08.401215 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038dbacc-3614-4c57-99a3-a76a4379515a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:08.403587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 138/284] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:21:08 2022] GET /identity/v3/users/2d2917634d1846bcaed69120a3386c81 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:08.409211 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c662249-7eb4-44b7-b007-591b30907267 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:08.535276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c662249-7eb4-44b7-b007-591b30907267 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:08.564747 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:08.574539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e983a-f87c-4032-aa90-2f16a0129ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:08.575418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e983a-f87c-4032-aa90-2f16a0129ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:08.575717 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e983a-f87c-4032-aa90-2f16a0129ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:08.576006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e983a-f87c-4032-aa90-2f16a0129ca2 None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:08.577240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e983a-f87c-4032-aa90-2f16a0129ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:08.578693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e983a-f87c-4032-aa90-2f16a0129ca2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:08.580721 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 147/285] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:08 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:08.587124 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dbacc-3614-4c57-99a3-a76a4379515a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:08.696602 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:08.727582 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dbacc-3614-4c57-99a3-a76a4379515a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:08.776332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb99210-a14f-46bd-9587-6d0126aad977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:08.777283 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb99210-a14f-46bd-9587-6d0126aad977 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:08.777573 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb99210-a14f-46bd-9587-6d0126aad977 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:08.777858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb99210-a14f-46bd-9587-6d0126aad977 None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10/users/2d2917634d1846bcaed69120a3386c81/roles/1026857a0bd64410a463448b53532814` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:08.779848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb99210-a14f-46bd-9587-6d0126aad977 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8a10b282fb412fbc79062901f84a10, user_id=2d2917634d1846bcaed69120a3386c81, role_id=1026857a0bd64410a463448b53532814)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:08.781332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb99210-a14f-46bd-9587-6d0126aad977 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:08.862890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 139/286] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:21:08 2022] PUT /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10/users/2d2917634d1846bcaed69120a3386c81/roles/1026857a0bd64410a463448b53532814 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:21:08.915856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7955787-c119-42b1-9f53-3fa4e702b862 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:08.916364 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7955787-c119-42b1-9f53-3fa4e702b862 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:08.916438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7955787-c119-42b1-9f53-3fa4e702b862 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:08.918072 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 148/287] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:08.923836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25404bb-ca5e-4fca-87df-967ea5e3bce2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:08.924135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25404bb-ca5e-4fca-87df-967ea5e3bce2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:08.924428 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25404bb-ca5e-4fca-87df-967ea5e3bce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:08.942430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b25404bb-ca5e-4fca-87df-967ea5e3bce2 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:08.944281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b25404bb-ca5e-4fca-87df-967ea5e3bce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:09.036958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 140/288] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:09.063092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7955787-c119-42b1-9f53-3fa4e702b862 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:09.203933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7955787-c119-42b1-9f53-3fa4e702b862 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:09.248776 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8f822-3386-410a-a643-b6d18a81fd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:09.249856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8f822-3386-410a-a643-b6d18a81fd25 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:09.250292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8f822-3386-410a-a643-b6d18a81fd25 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:09.250741 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8f822-3386-410a-a643-b6d18a81fd25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:09.265398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8f822-3386-410a-a643-b6d18a81fd25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-dfe8f822-3386-410a-a643-b6d18a81fd25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:09.274261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:09.275450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:09.276463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:09.276463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:09.276463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:09.276463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:09.276463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:09.276463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:09.276463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:09.276463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:09.276463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:09.276463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:09.276463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:09.276463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:09.278915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 149/289] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:09.284865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25404bb-ca5e-4fca-87df-967ea5e3bce2 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:09.382650 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:09.409154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25404bb-ca5e-4fca-87df-967ea5e3bce2 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:09.447959 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c6d12-1e4b-4fbe-ba19-fd4a492867b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:09.448884 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c6d12-1e4b-4fbe-ba19-fd4a492867b8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:09.449348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c6d12-1e4b-4fbe-ba19-fd4a492867b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:09.449908 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c6d12-1e4b-4fbe-ba19-fd4a492867b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:09.450873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c6d12-1e4b-4fbe-ba19-fd4a492867b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:09.451338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c6d12-1e4b-4fbe-ba19-fd4a492867b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:09.453165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c6d12-1e4b-4fbe-ba19-fd4a492867b8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:09.462769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 141/290] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:09.468479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8f822-3386-410a-a643-b6d18a81fd25 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:09.534587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:09.604428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8f822-3386-410a-a643-b6d18a81fd25 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:09.652487 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae380e8-0ba3-4b07-b797-f7522d22b378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:09.653355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae380e8-0ba3-4b07-b797-f7522d22b378 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:09.653650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae380e8-0ba3-4b07-b797-f7522d22b378 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:09.653929 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae380e8-0ba3-4b07-b797-f7522d22b378 None admin] PATH_INFO: `/v3/users/60a70ee64c114c609ad233fdbd782cc2` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:09.655302 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae380e8-0ba3-4b07-b797-f7522d22b378 None admin] RBAC: Authorizing `identity:get_user(user_id=60a70ee64c114c609ad233fdbd782cc2)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:09.656809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae380e8-0ba3-4b07-b797-f7522d22b378 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:09.659016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 150/291] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:21:09 2022] GET /identity/v3/users/60a70ee64c114c609ad233fdbd782cc2 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:09.664896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c6d12-1e4b-4fbe-ba19-fd4a492867b8 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:09.784452 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c6d12-1e4b-4fbe-ba19-fd4a492867b8 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:09.822586 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2f53e-4551-46f9-b3f0-eb0bd81e249b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:09.823253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2f53e-4551-46f9-b3f0-eb0bd81e249b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:09.823540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2f53e-4551-46f9-b3f0-eb0bd81e249b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:09.823820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2f53e-4551-46f9-b3f0-eb0bd81e249b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:09.832711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:09.895405 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee2f53e-4551-46f9-b3f0-eb0bd81e249b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:09.897536 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee2f53e-4551-46f9-b3f0-eb0bd81e249b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:10.001632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 142/292] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:10.010141 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae380e8-0ba3-4b07-b797-f7522d22b378 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:10.132164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae380e8-0ba3-4b07-b797-f7522d22b378 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:10.157601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:10.176264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8193503-f281-4794-b145-3f18f88bafdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:10.177171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8193503-f281-4794-b145-3f18f88bafdf None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:10.177465 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8193503-f281-4794-b145-3f18f88bafdf None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:10.177850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8193503-f281-4794-b145-3f18f88bafdf None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:10.179454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8193503-f281-4794-b145-3f18f88bafdf None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:10.180890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8193503-f281-4794-b145-3f18f88bafdf None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:10.182833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 151/293] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:10 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:10.195219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2f53e-4551-46f9-b3f0-eb0bd81e249b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:10.252535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:10.324150 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2f53e-4551-46f9-b3f0-eb0bd81e249b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:10.360350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aadfc9-431e-471c-b0bc-5b4826139ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:10.361015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aadfc9-431e-471c-b0bc-5b4826139ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:10.361334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aadfc9-431e-471c-b0bc-5b4826139ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:10.361612 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aadfc9-431e-471c-b0bc-5b4826139ac7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:10.362455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aadfc9-431e-471c-b0bc-5b4826139ac7 None admin] RBAC: Adding query filter params (role.id=9330473f62ec4a82a82d1c4beb6160d7, scope.project.id=bc8a10b282fb412fbc79062901f84a10, user.id=60a70ee64c114c609ad233fdbd782cc2) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:10.362770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aadfc9-431e-471c-b0bc-5b4826139ac7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:10.364206 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aadfc9-431e-471c-b0bc-5b4826139ac7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:10.371620 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 143/294] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:21:10 2022] GET /identity/v3/role_assignments?user.id=60a70ee64c114c609ad233fdbd782cc2&scope.project.id=bc8a10b282fb412fbc79062901f84a10&role.id=9330473f62ec4a82a82d1c4beb6160d7 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:10.942925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f952f-0013-4a5d-8f1d-933cecf04c2a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:10.943883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f952f-0013-4a5d-8f1d-933cecf04c2a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:10.944061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f952f-0013-4a5d-8f1d-933cecf04c2a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:10.945659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 152/295] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:10.952367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b294671-4bdc-499a-83f6-2223901e3bbd None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:10.952652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b294671-4bdc-499a-83f6-2223901e3bbd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:10.952851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b294671-4bdc-499a-83f6-2223901e3bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:10.970093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-9b294671-4bdc-499a-83f6-2223901e3bbd None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:10.971315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9b294671-4bdc-499a-83f6-2223901e3bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:11.055983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 144/296] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:11.057917 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb223a-7046-4298-8177-dfc8ea996b8f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:11.058783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb223a-7046-4298-8177-dfc8ea996b8f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:11.059274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb223a-7046-4298-8177-dfc8ea996b8f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:11.061142 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 153/297] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:11.067873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f93203-7305-4753-8a80-c11470e880b5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:11.067873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f93203-7305-4753-8a80-c11470e880b5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:11.067873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f93203-7305-4753-8a80-c11470e880b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:11.076596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb223a-7046-4298-8177-dfc8ea996b8f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:11.091987 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-90f93203-7305-4753-8a80-c11470e880b5 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:11.093513 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-90f93203-7305-4753-8a80-c11470e880b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:11.132292 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:11.176605 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 145/298] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:11.179630 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a4113-0a5b-4ec6-a644-7eea67d0d9c1 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:11.180411 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a4113-0a5b-4ec6-a644-7eea67d0d9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:11.180879 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a4113-0a5b-4ec6-a644-7eea67d0d9c1 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:11.182621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 146/299] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:11.187644 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653303a4-711c-4235-8450-6e1fc2dfcc3d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:11.187931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653303a4-711c-4235-8450-6e1fc2dfcc3d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:11.188209 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653303a4-711c-4235-8450-6e1fc2dfcc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:11.203628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb223a-7046-4298-8177-dfc8ea996b8f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:11.208184 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-653303a4-711c-4235-8450-6e1fc2dfcc3d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:11.209441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-653303a4-711c-4235-8450-6e1fc2dfcc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:11.241867 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111aed39-fc8a-4fbd-8a4d-c5ab43980dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:11.242671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111aed39-fc8a-4fbd-8a4d-c5ab43980dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:11.242960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111aed39-fc8a-4fbd-8a4d-c5ab43980dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:11.243266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111aed39-fc8a-4fbd-8a4d-c5ab43980dc6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:11.254300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111aed39-fc8a-4fbd-8a4d-c5ab43980dc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-111aed39-fc8a-4fbd-8a4d-c5ab43980dc6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:11.262869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:11.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:11.264660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:11.264660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:11.264660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:11.264660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:11.264660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:11.264660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:11.264660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:11.264660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:11.264660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:11.264660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:11.264660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:21:11.264660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:11.268835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 154/300] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:21:11.269758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111aed39-fc8a-4fbd-8a4d-c5ab43980dc6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:11.280038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 147/301] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:11.289955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653303a4-711c-4235-8450-6e1fc2dfcc3d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:11.323166 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:11.374099 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111aed39-fc8a-4fbd-8a4d-c5ab43980dc6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:11.389981 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:11.399215 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653303a4-711c-4235-8450-6e1fc2dfcc3d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:11.411661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89010d-b571-4938-9304-3976bfaf0ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:11.411661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89010d-b571-4938-9304-3976bfaf0ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:11.411661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89010d-b571-4938-9304-3976bfaf0ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:11.411661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89010d-b571-4938-9304-3976bfaf0ac6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:11.412644 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89010d-b571-4938-9304-3976bfaf0ac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:11.413337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89010d-b571-4938-9304-3976bfaf0ac6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:11.417229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 155/302] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:11.417229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89010d-b571-4938-9304-3976bfaf0ac6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:11.441046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a6c0a-624b-45a4-91c5-97bc70866293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:11.441742 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a6c0a-624b-45a4-91c5-97bc70866293 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:11.442026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a6c0a-624b-45a4-91c5-97bc70866293 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:11.442303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a6c0a-624b-45a4-91c5-97bc70866293 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:11.443048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a6c0a-624b-45a4-91c5-97bc70866293 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:11.444250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a6c0a-624b-45a4-91c5-97bc70866293 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:11.445404 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a6c0a-624b-45a4-91c5-97bc70866293 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:11.454462 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 148/303] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:11.455198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a6c0a-624b-45a4-91c5-97bc70866293 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:11.527349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89010d-b571-4938-9304-3976bfaf0ac6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:11.544577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:11.559664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a6c0a-624b-45a4-91c5-97bc70866293 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:11.561902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a51492-4929-4483-b961-33a2a65108e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:11.562713 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a51492-4929-4483-b961-33a2a65108e8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:11.563041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a51492-4929-4483-b961-33a2a65108e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:11.563326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a51492-4929-4483-b961-33a2a65108e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:11.575693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a51492-4929-4483-b961-33a2a65108e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-28a51492-4929-4483-b961-33a2a65108e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:11.584807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:11.586043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:11.587042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:11.587042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:11.587042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:11.587042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:11.587042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:11.587042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:11.587042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:11.587042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:11.587042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:11.587042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:11.587042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:11.587042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:11.587954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 156/304] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:11.588796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a51492-4929-4483-b961-33a2a65108e8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:11.604026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17730113-fc62-4413-b86a-157b09d39b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:11.604939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17730113-fc62-4413-b86a-157b09d39b9e None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:11.605409 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17730113-fc62-4413-b86a-157b09d39b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:11.605844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17730113-fc62-4413-b86a-157b09d39b9e None admin] PATH_INFO: `/v3/groups` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:11.606679 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17730113-fc62-4413-b86a-157b09d39b9e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:11.608283 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17730113-fc62-4413-b86a-157b09d39b9e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:11.691433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a51492-4929-4483-b961-33a2a65108e8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:11.708003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 149/305] 10.222.0.64 () {60 vars in 1134 bytes} [Mon May 23 15:21:11 2022] POST /identity/v3/groups => generated 229 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:21:11.710616 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17730113-fc62-4413-b86a-157b09d39b9e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:11.732894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857f79e-4079-4598-8d1f-bb32bb2fe0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:11.734182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857f79e-4079-4598-8d1f-bb32bb2fe0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:11.734747 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857f79e-4079-4598-8d1f-bb32bb2fe0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:11.735214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857f79e-4079-4598-8d1f-bb32bb2fe0d6 None admin] PATH_INFO: `/v3/users/7d5f6895ba4b40d9a278b39d7ffbe782` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:11.737494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:11.755357 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857f79e-4079-4598-8d1f-bb32bb2fe0d6 None admin] RBAC: Authorizing `identity:get_user(user_id=7d5f6895ba4b40d9a278b39d7ffbe782)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:11.757824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857f79e-4079-4598-8d1f-bb32bb2fe0d6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:11.761204 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 157/306] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:21:11 2022] GET /identity/v3/users/7d5f6895ba4b40d9a278b39d7ffbe782 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:11.767026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857f79e-4079-4598-8d1f-bb32bb2fe0d6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:11.818177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:11.866127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17730113-fc62-4413-b86a-157b09d39b9e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:11.904163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857f79e-4079-4598-8d1f-bb32bb2fe0d6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:11.905624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ace9e1-1dd1-4984-8422-5a4cc0d0ff1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:11.907406 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ace9e1-1dd1-4984-8422-5a4cc0d0ff1f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:11.907921 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ace9e1-1dd1-4984-8422-5a4cc0d0ff1f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:11.908411 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ace9e1-1dd1-4984-8422-5a4cc0d0ff1f None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:11.909426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ace9e1-1dd1-4984-8422-5a4cc0d0ff1f None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:11.909962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ace9e1-1dd1-4984-8422-5a4cc0d0ff1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:11.911577 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ace9e1-1dd1-4984-8422-5a4cc0d0ff1f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:11.920898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 150/307] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:11.926776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ace9e1-1dd1-4984-8422-5a4cc0d0ff1f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:11.954764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69158414-d3f7-45e8-aaae-3dcf8357e74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:11.955428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69158414-d3f7-45e8-aaae-3dcf8357e74a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:11.955715 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69158414-d3f7-45e8-aaae-3dcf8357e74a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:11.955994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69158414-d3f7-45e8-aaae-3dcf8357e74a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:11.956827 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69158414-d3f7-45e8-aaae-3dcf8357e74a None admin] RBAC: Adding query filter params (role.id=1026857a0bd64410a463448b53532814, user.id=7d5f6895ba4b40d9a278b39d7ffbe782, scope.system=all) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:11.957175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69158414-d3f7-45e8-aaae-3dcf8357e74a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:11.959437 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69158414-d3f7-45e8-aaae-3dcf8357e74a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:11.966060 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 158/308] 10.222.0.64 () {56 vars in 1308 bytes} [Mon May 23 15:21:11 2022] GET /identity/v3/role_assignments?user.id=7d5f6895ba4b40d9a278b39d7ffbe782&scope.system=all&role.id=1026857a0bd64410a463448b53532814 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:12.032494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:12.060127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ace9e1-1dd1-4984-8422-5a4cc0d0ff1f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:12.062747 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23279c65-5d55-4923-896c-f7fe094cf2f3 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:12.063246 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23279c65-5d55-4923-896c-f7fe094cf2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:12.063332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23279c65-5d55-4923-896c-f7fe094cf2f3 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:12.065177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 159/309] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:12.071724 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85acf9-9f05-4b56-99ed-269da9d7b49c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:12.072077 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85acf9-9f05-4b56-99ed-269da9d7b49c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:12.072444 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85acf9-9f05-4b56-99ed-269da9d7b49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:12.091006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0b85acf9-9f05-4b56-99ed-269da9d7b49c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:12.092335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0b85acf9-9f05-4b56-99ed-269da9d7b49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:12.105096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b6e3e-92d4-4abc-b6e3-abc3d2e3f233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:12.106446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b6e3e-92d4-4abc-b6e3-abc3d2e3f233 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:12.106734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b6e3e-92d4-4abc-b6e3-abc3d2e3f233 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:12.107297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b6e3e-92d4-4abc-b6e3-abc3d2e3f233 None admin] PATH_INFO: `/v3/users/8fce566f37654115a97c394065c7d7fc` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:12.108856 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b6e3e-92d4-4abc-b6e3-abc3d2e3f233 None admin] RBAC: Authorizing `identity:get_user(user_id=8fce566f37654115a97c394065c7d7fc)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:12.111143 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b6e3e-92d4-4abc-b6e3-abc3d2e3f233 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:12.114030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 151/310] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:21:11 2022] GET /identity/v3/users/8fce566f37654115a97c394065c7d7fc => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:12.118830 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b6e3e-92d4-4abc-b6e3-abc3d2e3f233 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:12.178539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 160/311] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:21:12.203424 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85acf9-9f05-4b56-99ed-269da9d7b49c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:12.245688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b6e3e-92d4-4abc-b6e3-abc3d2e3f233 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:12.277457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:12.283115 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3ed68-453e-4c7d-b4b2-316afd825b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:12.284246 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3ed68-453e-4c7d-b4b2-316afd825b90 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:12.284735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3ed68-453e-4c7d-b4b2-316afd825b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:12.285304 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3ed68-453e-4c7d-b4b2-316afd825b90 None admin] PATH_INFO: `/v3/system/users/8fce566f37654115a97c394065c7d7fc/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:12.287519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3ed68-453e-4c7d-b4b2-316afd825b90 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8fce566f37654115a97c394065c7d7fc, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:12.289699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3ed68-453e-4c7d-b4b2-316afd825b90 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:12.309470 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85acf9-9f05-4b56-99ed-269da9d7b49c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:12.346913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d7910-313e-4a4b-b6ee-523a74ee9a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:12.347740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d7910-313e-4a4b-b6ee-523a74ee9a8e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:12.348025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d7910-313e-4a4b-b6ee-523a74ee9a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:12.348335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d7910-313e-4a4b-b6ee-523a74ee9a8e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:12.359732 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d7910-313e-4a4b-b6ee-523a74ee9a8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-843d7910-313e-4a4b-b6ee-523a74ee9a8e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:12.368251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:12.369216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:12.374889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:12.374889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:12.374889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:12.374889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:12.374889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:12.374889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:12.374889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:12.374889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:12.374889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:12.374889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:12.374889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:21:12.374889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:12.374889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 161/312] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:21:12.375564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d7910-313e-4a4b-b6ee-523a74ee9a8e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:12.378074 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 152/313] 10.222.0.64 () {58 vars in 1334 bytes} [Mon May 23 15:21:12 2022] PUT /identity/v3/system/users/8fce566f37654115a97c394065c7d7fc/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:21:12.509858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:12.522168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d7910-313e-4a4b-b6ee-523a74ee9a8e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:12.561849 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da43740-5e49-4cca-b47e-8a2b3406e516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:12.562685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da43740-5e49-4cca-b47e-8a2b3406e516 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:12.563224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da43740-5e49-4cca-b47e-8a2b3406e516 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:12.563602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da43740-5e49-4cca-b47e-8a2b3406e516 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:12.564449 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da43740-5e49-4cca-b47e-8a2b3406e516 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:12.564872 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da43740-5e49-4cca-b47e-8a2b3406e516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:12.566380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da43740-5e49-4cca-b47e-8a2b3406e516 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:12.575952 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 162/314] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:12.581121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3ed68-453e-4c7d-b4b2-316afd825b90 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:12.594324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:12.693026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3ed68-453e-4c7d-b4b2-316afd825b90 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:12.732039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db86fa-28e9-44ba-a040-350d40ccad26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:12.732928 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db86fa-28e9-44ba-a040-350d40ccad26 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:12.733263 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db86fa-28e9-44ba-a040-350d40ccad26 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:12.733548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db86fa-28e9-44ba-a040-350d40ccad26 None admin] PATH_INFO: `/v3/users/2d2917634d1846bcaed69120a3386c81` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:12.734755 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db86fa-28e9-44ba-a040-350d40ccad26 None admin] RBAC: Authorizing `identity:get_user(user_id=2d2917634d1846bcaed69120a3386c81)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:12.736231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db86fa-28e9-44ba-a040-350d40ccad26 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:12.738254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 153/315] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:21:12 2022] GET /identity/v3/users/2d2917634d1846bcaed69120a3386c81 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:12.744434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da43740-5e49-4cca-b47e-8a2b3406e516 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:12.852594 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da43740-5e49-4cca-b47e-8a2b3406e516 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:12.887482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3f86b-fb4e-4899-9cd1-368152af3458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:12.888400 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3f86b-fb4e-4899-9cd1-368152af3458 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:12.888828 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3f86b-fb4e-4899-9cd1-368152af3458 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:12.889416 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3f86b-fb4e-4899-9cd1-368152af3458 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:12.910983 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:12.961752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3f86b-fb4e-4899-9cd1-368152af3458 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:12.963281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3f86b-fb4e-4899-9cd1-368152af3458 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:13.063159 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 163/316] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:13.069443 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db86fa-28e9-44ba-a040-350d40ccad26 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:13.150779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:13.176784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db86fa-28e9-44ba-a040-350d40ccad26 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:13.216929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfe8dc-597f-45c3-8f53-74ad37f1eee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:13.218214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfe8dc-597f-45c3-8f53-74ad37f1eee4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:13.218701 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfe8dc-597f-45c3-8f53-74ad37f1eee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:13.219179 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfe8dc-597f-45c3-8f53-74ad37f1eee4 None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:13.220593 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfe8dc-597f-45c3-8f53-74ad37f1eee4 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:13.222294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfe8dc-597f-45c3-8f53-74ad37f1eee4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:13.224623 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 154/317] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:13 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:13.230075 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3f86b-fb4e-4899-9cd1-368152af3458 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:13.284432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:13.334828 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3f86b-fb4e-4899-9cd1-368152af3458 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:13.374703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc826365-bd7d-4a99-853f-2febdb25ee06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:13.375777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc826365-bd7d-4a99-853f-2febdb25ee06 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:13.376249 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc826365-bd7d-4a99-853f-2febdb25ee06 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:13.376730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc826365-bd7d-4a99-853f-2febdb25ee06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:13.377773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc826365-bd7d-4a99-853f-2febdb25ee06 None admin] RBAC: Adding query filter params (role.id=1026857a0bd64410a463448b53532814, scope.project.id=bc8a10b282fb412fbc79062901f84a10, user.id=2d2917634d1846bcaed69120a3386c81) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:13.378091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc826365-bd7d-4a99-853f-2febdb25ee06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:13.379616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc826365-bd7d-4a99-853f-2febdb25ee06 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:13.386466 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 164/318] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:21:13 2022] GET /identity/v3/role_assignments?user.id=2d2917634d1846bcaed69120a3386c81&scope.project.id=bc8a10b282fb412fbc79062901f84a10&role.id=1026857a0bd64410a463448b53532814 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:21:14.043164 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06f501-fa74-4cfe-a5cf-6f0cfcb3a8f6 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:14.043670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06f501-fa74-4cfe-a5cf-6f0cfcb3a8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:14.043774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06f501-fa74-4cfe-a5cf-6f0cfcb3a8f6 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:14.045243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 155/319] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:14.050236 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9772ab-c9af-4dc1-bb5f-3f76e716d1cd None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:14.050528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9772ab-c9af-4dc1-bb5f-3f76e716d1cd None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:14.050805 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9772ab-c9af-4dc1-bb5f-3f76e716d1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:14.068348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7b9772ab-c9af-4dc1-bb5f-3f76e716d1cd None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:14.069544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7b9772ab-c9af-4dc1-bb5f-3f76e716d1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:14.139751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 165/320] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:14.162902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06f501-fa74-4cfe-a5cf-6f0cfcb3a8f6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:14.265512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06f501-fa74-4cfe-a5cf-6f0cfcb3a8f6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:14.299721 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aad54e-2409-46bc-a398-f288dc4d6a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:14.300445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aad54e-2409-46bc-a398-f288dc4d6a7f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:14.300728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aad54e-2409-46bc-a398-f288dc4d6a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:14.301002 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aad54e-2409-46bc-a398-f288dc4d6a7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:14.302169 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aad54e-2409-46bc-a398-f288dc4d6a7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:14.304193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aad54e-2409-46bc-a398-f288dc4d6a7f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:14.305993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 156/321] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:14.311353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9772ab-c9af-4dc1-bb5f-3f76e716d1cd None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:14.315299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1f264-1e0b-498a-adad-9dd745462136 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:14.315592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1f264-1e0b-498a-adad-9dd745462136 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:14.315898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1f264-1e0b-498a-adad-9dd745462136 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:14.317368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 157/322] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:21:14.322725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e449b53-9282-41da-8b0a-c3281e0df1dd None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:14.323028 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e449b53-9282-41da-8b0a-c3281e0df1dd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:14.323311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e449b53-9282-41da-8b0a-c3281e0df1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:14.341171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-5e449b53-9282-41da-8b0a-c3281e0df1dd None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:14.342604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5e449b53-9282-41da-8b0a-c3281e0df1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:14.351699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:14.360985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:14.412806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9772ab-c9af-4dc1-bb5f-3f76e716d1cd None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:14.414290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 158/323] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:21:14.437115 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e449b53-9282-41da-8b0a-c3281e0df1dd None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:14.448314 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791d28b-4147-4636-96f2-22c7ace2a1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:14.449273 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791d28b-4147-4636-96f2-22c7ace2a1b5 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:14.449557 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791d28b-4147-4636-96f2-22c7ace2a1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:14.449834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791d28b-4147-4636-96f2-22c7ace2a1b5 None admin] PATH_INFO: `/v3/groups` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:14.450536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791d28b-4147-4636-96f2-22c7ace2a1b5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:14.451975 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791d28b-4147-4636-96f2-22c7ace2a1b5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:14.540316 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e449b53-9282-41da-8b0a-c3281e0df1dd None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:14.569255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 166/324] 10.222.0.64 () {60 vars in 1134 bytes} [Mon May 23 15:21:14 2022] POST /identity/v3/groups => generated 226 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:21:14.589605 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1c07d-9838-48a0-be13-e8af894bb535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:14.590433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1c07d-9838-48a0-be13-e8af894bb535 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:14.590716 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1c07d-9838-48a0-be13-e8af894bb535 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:14.590995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1c07d-9838-48a0-be13-e8af894bb535 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:14.604420 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1c07d-9838-48a0-be13-e8af894bb535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-32c1c07d-9838-48a0-be13-e8af894bb535 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:14.612553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:14.613740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:14.614695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:14.614695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:14.614695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:14.614695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:14.614695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:14.614695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:14.614695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:14.614695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:14.614695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:14.614695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:14.614695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:21:14.614695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:14.614695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 159/325] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:21:14.620347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791d28b-4147-4636-96f2-22c7ace2a1b5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:14.743135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5e01a-b1a2-4966-9516-7169642d1343 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:14.743474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5e01a-b1a2-4966-9516-7169642d1343 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:14.743711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5e01a-b1a2-4966-9516-7169642d1343 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:14.745249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 160/326] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:21:14.750458 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6125ce-2a32-4b36-842f-37c7d0afb2b4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:14.750753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6125ce-2a32-4b36-842f-37c7d0afb2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:14.751041 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6125ce-2a32-4b36-842f-37c7d0afb2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:14.756745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:14.760678 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791d28b-4147-4636-96f2-22c7ace2a1b5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:14.768224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:14.773736 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6a6125ce-2a32-4b36-842f-37c7d0afb2b4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:14.774956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6a6125ce-2a32-4b36-842f-37c7d0afb2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:14.802241 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc13cb5-2ab1-4058-9baf-301666eca1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:14.802972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc13cb5-2ab1-4058-9baf-301666eca1ee None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:14.803519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc13cb5-2ab1-4058-9baf-301666eca1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:14.803803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc13cb5-2ab1-4058-9baf-301666eca1ee None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:14.804560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc13cb5-2ab1-4058-9baf-301666eca1ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:14.804875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc13cb5-2ab1-4058-9baf-301666eca1ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:14.806299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc13cb5-2ab1-4058-9baf-301666eca1ee None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:14.816183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 167/327] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:14.823712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc13cb5-2ab1-4058-9baf-301666eca1ee None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:14.860695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 161/328] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:21:14.885341 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6125ce-2a32-4b36-842f-37c7d0afb2b4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:14.945483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc13cb5-2ab1-4058-9baf-301666eca1ee None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:14.992377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7668ad8-5c7f-48fc-9944-fbdfa65654f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:14.993317 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7668ad8-5c7f-48fc-9944-fbdfa65654f5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:14.993603 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7668ad8-5c7f-48fc-9944-fbdfa65654f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:14.993879 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7668ad8-5c7f-48fc-9944-fbdfa65654f5 None admin] PATH_INFO: `/v3/users/7d5f6895ba4b40d9a278b39d7ffbe782` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:14.995095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7668ad8-5c7f-48fc-9944-fbdfa65654f5 None admin] RBAC: Authorizing `identity:get_user(user_id=7d5f6895ba4b40d9a278b39d7ffbe782)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:14.996528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7668ad8-5c7f-48fc-9944-fbdfa65654f5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:14.998483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 168/329] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:21:14 2022] GET /identity/v3/users/7d5f6895ba4b40d9a278b39d7ffbe782 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:15.003998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7668ad8-5c7f-48fc-9944-fbdfa65654f5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:15.023956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6125ce-2a32-4b36-842f-37c7d0afb2b4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:15.048458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:15.062315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:15.065084 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10bf08-5aed-4c83-98d4-f56cd6a1a92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:15.065906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10bf08-5aed-4c83-98d4-f56cd6a1a92c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:15.066811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10bf08-5aed-4c83-98d4-f56cd6a1a92c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:15.066811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10bf08-5aed-4c83-98d4-f56cd6a1a92c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:15.086905 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10bf08-5aed-4c83-98d4-f56cd6a1a92c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-5c10bf08-5aed-4c83-98d4-f56cd6a1a92c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:15.093350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:15.094274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:15.095095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:15.095095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:15.095095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:15.095095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:15.095095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:15.095095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:15.095095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:15.095095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:15.095095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:15.095095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:15.095095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:15.095095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:15.095095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 162/330] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:15.100551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10bf08-5aed-4c83-98d4-f56cd6a1a92c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:15.136884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7668ad8-5c7f-48fc-9944-fbdfa65654f5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:15.194014 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2d6e1-830e-45d2-b2bd-99afa3b61852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:15.195422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2d6e1-830e-45d2-b2bd-99afa3b61852 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:15.196697 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2d6e1-830e-45d2-b2bd-99afa3b61852 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:15.197183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2d6e1-830e-45d2-b2bd-99afa3b61852 None admin] PATH_INFO: `/v3/system/users/7d5f6895ba4b40d9a278b39d7ffbe782/roles/1026857a0bd64410a463448b53532814` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:15.198903 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2d6e1-830e-45d2-b2bd-99afa3b61852 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7d5f6895ba4b40d9a278b39d7ffbe782, role_id=1026857a0bd64410a463448b53532814)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:15.200446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2d6e1-830e-45d2-b2bd-99afa3b61852 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:15.231439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10bf08-5aed-4c83-98d4-f56cd6a1a92c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:15.270166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58394db2-5da8-432e-9c42-a102fef3c025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:15.270894 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58394db2-5da8-432e-9c42-a102fef3c025 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:15.271219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58394db2-5da8-432e-9c42-a102fef3c025 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:15.271503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58394db2-5da8-432e-9c42-a102fef3c025 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:15.272258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58394db2-5da8-432e-9c42-a102fef3c025 None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:15.272571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58394db2-5da8-432e-9c42-a102fef3c025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:15.273996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58394db2-5da8-432e-9c42-a102fef3c025 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:15.280550 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 169/331] 10.222.0.64 () {58 vars in 1334 bytes} [Mon May 23 15:21:15 2022] PUT /identity/v3/system/users/7d5f6895ba4b40d9a278b39d7ffbe782/roles/1026857a0bd64410a463448b53532814 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:21:15.288279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 163/332] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:15.294652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2d6e1-830e-45d2-b2bd-99afa3b61852 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:15.423001 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:15.429326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2d6e1-830e-45d2-b2bd-99afa3b61852 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:15.472891 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b9451-773f-41d6-9216-e3acde70635f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:15.473928 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b9451-773f-41d6-9216-e3acde70635f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:15.474779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b9451-773f-41d6-9216-e3acde70635f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:15.475091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b9451-773f-41d6-9216-e3acde70635f None admin] PATH_INFO: `/v3/users/8fce566f37654115a97c394065c7d7fc` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:15.478454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b9451-773f-41d6-9216-e3acde70635f None admin] RBAC: Authorizing `identity:get_user(user_id=8fce566f37654115a97c394065c7d7fc)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:15.479948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b9451-773f-41d6-9216-e3acde70635f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:15.482189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 170/333] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:21:15 2022] GET /identity/v3/users/8fce566f37654115a97c394065c7d7fc => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:15.488349 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58394db2-5da8-432e-9c42-a102fef3c025 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:15.551163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:15.603314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58394db2-5da8-432e-9c42-a102fef3c025 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:15.641079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2960040-e212-409d-9c84-1780cd4bb1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:15.641819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2960040-e212-409d-9c84-1780cd4bb1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:15.642110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2960040-e212-409d-9c84-1780cd4bb1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:15.642389 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2960040-e212-409d-9c84-1780cd4bb1e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:15.643279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2960040-e212-409d-9c84-1780cd4bb1e4 None admin] RBAC: Adding query filter params (role.id=9330473f62ec4a82a82d1c4beb6160d7, user.id=8fce566f37654115a97c394065c7d7fc, scope.system=all) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:15.643593 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2960040-e212-409d-9c84-1780cd4bb1e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:15.645029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2960040-e212-409d-9c84-1780cd4bb1e4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:15.658532 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 164/334] 10.222.0.64 () {56 vars in 1308 bytes} [Mon May 23 15:21:15 2022] GET /identity/v3/role_assignments?user.id=8fce566f37654115a97c394065c7d7fc&scope.system=all&role.id=9330473f62ec4a82a82d1c4beb6160d7 => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:17.337731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b5653-4c6d-41e0-a831-d05c7673185a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:17.338270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b5653-4c6d-41e0-a831-d05c7673185a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:17.338355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b5653-4c6d-41e0-a831-d05c7673185a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:17.339943 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 171/335] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:17.345165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b3da3-22f6-4711-9cde-c0368003146e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:17.345611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b3da3-22f6-4711-9cde-c0368003146e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:17.345791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b3da3-22f6-4711-9cde-c0368003146e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:17.365871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c07b3da3-22f6-4711-9cde-c0368003146e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:17.367301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c07b3da3-22f6-4711-9cde-c0368003146e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:17.452038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 165/336] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:17.487792 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b5653-4c6d-41e0-a831-d05c7673185a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:17.600234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b5653-4c6d-41e0-a831-d05c7673185a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:17.642366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799381ca-33b4-4c0a-9bab-453a55f0d1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:17.643187 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799381ca-33b4-4c0a-9bab-453a55f0d1de None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:17.643516 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799381ca-33b4-4c0a-9bab-453a55f0d1de None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:17.643803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799381ca-33b4-4c0a-9bab-453a55f0d1de None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:17.660763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799381ca-33b4-4c0a-9bab-453a55f0d1de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-799381ca-33b4-4c0a-9bab-453a55f0d1de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:17.674027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:17.674965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:17.676407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:17.676407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:17.676407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:17.676407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:17.676407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:17.676407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:17.676407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:17.676407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:17.676407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:17.676407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:17.676407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:17.676407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:17.676407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 172/337] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:17.681525 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b3da3-22f6-4711-9cde-c0368003146e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:17.736479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:17.801687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b3da3-22f6-4711-9cde-c0368003146e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:17.853420 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1052ee-86cb-44a6-a63f-fc27c02c3bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:17.854451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1052ee-86cb-44a6-a63f-fc27c02c3bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:17.854975 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1052ee-86cb-44a6-a63f-fc27c02c3bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:17.855455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1052ee-86cb-44a6-a63f-fc27c02c3bd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:17.856425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1052ee-86cb-44a6-a63f-fc27c02c3bd7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:17.856947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1052ee-86cb-44a6-a63f-fc27c02c3bd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:17.858573 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1052ee-86cb-44a6-a63f-fc27c02c3bd7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:17.872346 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 166/338] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:17.873736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799381ca-33b4-4c0a-9bab-453a55f0d1de None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:17.919201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:18.019624 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799381ca-33b4-4c0a-9bab-453a55f0d1de None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:18.063193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7732a9f-47cf-4086-a0a8-b279a2ef03a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:18.063853 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7732a9f-47cf-4086-a0a8-b279a2ef03a0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:18.064140 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7732a9f-47cf-4086-a0a8-b279a2ef03a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:18.064421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7732a9f-47cf-4086-a0a8-b279a2ef03a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:18.147960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7732a9f-47cf-4086-a0a8-b279a2ef03a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:18.147960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7732a9f-47cf-4086-a0a8-b279a2ef03a0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:18.190203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:18.285878 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 173/339] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:18.291785 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1052ee-86cb-44a6-a63f-fc27c02c3bd7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:18.413139 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1052ee-86cb-44a6-a63f-fc27c02c3bd7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:18.415795 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:18.464535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596737d-8d46-473d-baf2-ffedf72b7f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:18.465456 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596737d-8d46-473d-baf2-ffedf72b7f39 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:18.465745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596737d-8d46-473d-baf2-ffedf72b7f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:18.466028 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596737d-8d46-473d-baf2-ffedf72b7f39 None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:18.467246 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a596737d-8d46-473d-baf2-ffedf72b7f39 None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:18.468708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a596737d-8d46-473d-baf2-ffedf72b7f39 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:18.470810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 167/340] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:18 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:18.476604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7732a9f-47cf-4086-a0a8-b279a2ef03a0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:18.605511 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7732a9f-47cf-4086-a0a8-b279a2ef03a0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:18.610722 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c92ce1-b6d8-48c7-b6f6-75405c50afd7 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:18.611014 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c92ce1-b6d8-48c7-b6f6-75405c50afd7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:18.611304 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c92ce1-b6d8-48c7-b6f6-75405c50afd7 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:18.614118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 168/341] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:21:18.621126 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd41e98-68af-4060-abc3-31d5b6077829 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:18.621428 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd41e98-68af-4060-abc3-31d5b6077829 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:18.621714 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd41e98-68af-4060-abc3-31d5b6077829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:18.648424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-efd41e98-68af-4060-abc3-31d5b6077829 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:18.650575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:18.653742 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-efd41e98-68af-4060-abc3-31d5b6077829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:18.669894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4db16b-dfff-4ce0-99ab-5ba287367237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:18.670998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4db16b-dfff-4ce0-99ab-5ba287367237 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:18.671424 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4db16b-dfff-4ce0-99ab-5ba287367237 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:18.671953 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4db16b-dfff-4ce0-99ab-5ba287367237 None admin] PATH_INFO: `/v3/groups/a83da4807fb249adb95a024797878b3b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:18.683638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4db16b-dfff-4ce0-99ab-5ba287367237 None admin] RBAC: Authorizing `identity:get_group(group_id=a83da4807fb249adb95a024797878b3b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:18.685350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4db16b-dfff-4ce0-99ab-5ba287367237 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:18.687558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 174/342] 10.222.0.64 () {56 vars in 1180 bytes} [Mon May 23 15:21:18 2022] GET /identity/v3/groups/a83da4807fb249adb95a024797878b3b => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:18.690886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e29d4-25ff-45d5-bce6-b943d807c7b4 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:18.691362 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e29d4-25ff-45d5-bce6-b943d807c7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:18.691785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e29d4-25ff-45d5-bce6-b943d807c7b4 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:18.693484 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 175/343] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:21:18.694373 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e29d4-25ff-45d5-bce6-b943d807c7b4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:18.815481 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 169/344] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:21:18.816942 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5bc69-1126-4975-beea-063229818442 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:18.818472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5bc69-1126-4975-beea-063229818442 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:18.818775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5bc69-1126-4975-beea-063229818442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:18.824265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e29d4-25ff-45d5-bce6-b943d807c7b4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:18.846777 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e3e5bc69-1126-4975-beea-063229818442 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:18.847980 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e3e5bc69-1126-4975-beea-063229818442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:18.864307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c00313-7cdc-453c-a590-06b06ade3bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:18.865992 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c00313-7cdc-453c-a590-06b06ade3bdb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:18.866287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c00313-7cdc-453c-a590-06b06ade3bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:18.866569 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c00313-7cdc-453c-a590-06b06ade3bdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:18.867407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c00313-7cdc-453c-a590-06b06ade3bdb None admin] RBAC: Adding query filter params (group.id=a83da4807fb249adb95a024797878b3b, role.id=9330473f62ec4a82a82d1c4beb6160d7, scope.project.id=936684d8d665414698e461a87200ef9b) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:18.867783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c00313-7cdc-453c-a590-06b06ade3bdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:18.869353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c00313-7cdc-453c-a590-06b06ade3bdb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:18.875791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 176/345] 10.222.0.64 () {56 vars in 1376 bytes} [Mon May 23 15:21:18 2022] GET /identity/v3/role_assignments?group.id=a83da4807fb249adb95a024797878b3b&scope.project.id=936684d8d665414698e461a87200ef9b&role.id=9330473f62ec4a82a82d1c4beb6160d7 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:18.876858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c00313-7cdc-453c-a590-06b06ade3bdb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:18.983216 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 170/346] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:18.984872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c19667-e36d-4a12-89c9-48bad80fc077 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:18.985191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c19667-e36d-4a12-89c9-48bad80fc077 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:18.985479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c19667-e36d-4a12-89c9-48bad80fc077 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:18.987439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 171/347] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:18.993998 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4261d4-0031-4499-a98e-419f45ae7efc None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:18.994386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4261d4-0031-4499-a98e-419f45ae7efc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:18.994675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4261d4-0031-4499-a98e-419f45ae7efc None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:18.996569 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 172/348] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:18.998701 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17428e3c-0a74-43de-977e-b67df1d07edd None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:18.999163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17428e3c-0a74-43de-977e-b67df1d07edd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:18.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17428e3c-0a74-43de-977e-b67df1d07edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:19.007873 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:19.023421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-17428e3c-0a74-43de-977e-b67df1d07edd None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:19.028338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-17428e3c-0a74-43de-977e-b67df1d07edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:19.061615 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:19.086784 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c00313-7cdc-453c-a590-06b06ade3bdb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:19.129165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 173/349] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:19.131397 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553b03d-dd2a-47b0-966b-6634ad361f76 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:19.131990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553b03d-dd2a-47b0-966b-6634ad361f76 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:19.132455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553b03d-dd2a-47b0-966b-6634ad361f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:19.135098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9c4d9-07ba-4204-82eb-16b24af087ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:19.136098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9c4d9-07ba-4204-82eb-16b24af087ae None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:19.136530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9c4d9-07ba-4204-82eb-16b24af087ae None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:19.136954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9c4d9-07ba-4204-82eb-16b24af087ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:19.150654 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9c4d9-07ba-4204-82eb-16b24af087ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:19.156420 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3553b03d-dd2a-47b0-966b-6634ad361f76 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:19.157740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3553b03d-dd2a-47b0-966b-6634ad361f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b0d9c4d9-07ba-4204-82eb-16b24af087ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:19.159539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:19.160564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:19.161417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:19.161417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:19.161417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:19.161417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:19.161417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:19.161417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:19.161417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:19.161417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:19.161417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:19.161417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:19.161417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:19.161417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:19.165428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 177/350] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:19.166416 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9c4d9-07ba-4204-82eb-16b24af087ae None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:19.246551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 174/351] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:19.247474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553b03d-dd2a-47b0-966b-6634ad361f76 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:19.305855 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9c4d9-07ba-4204-82eb-16b24af087ae None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:19.333903 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:19.352896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:19.363887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c05f1-27d3-4bd7-b7aa-0ac04aa3b7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:19.365564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c05f1-27d3-4bd7-b7aa-0ac04aa3b7fa None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:19.365908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c05f1-27d3-4bd7-b7aa-0ac04aa3b7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:19.366325 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c05f1-27d3-4bd7-b7aa-0ac04aa3b7fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:19.367557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553b03d-dd2a-47b0-966b-6634ad361f76 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:19.381023 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c05f1-27d3-4bd7-b7aa-0ac04aa3b7fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-4b7c05f1-27d3-4bd7-b7aa-0ac04aa3b7fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:19.393690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:19.394634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:19.395483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:19.395483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:19.395483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:19.395483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:19.395483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:19.395483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:19.395483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:19.395483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:19.395483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:19.395483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:19.395483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:19.395483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:19.395483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 178/352] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:21:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:19.400452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c05f1-27d3-4bd7-b7aa-0ac04aa3b7fa None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:19.418700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59999829-a593-49f1-a77c-ca128f16e2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:19.419362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59999829-a593-49f1-a77c-ca128f16e2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:19.419596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59999829-a593-49f1-a77c-ca128f16e2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:19.419883 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59999829-a593-49f1-a77c-ca128f16e2c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:19.420646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59999829-a593-49f1-a77c-ca128f16e2c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:19.420957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59999829-a593-49f1-a77c-ca128f16e2c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:19.422421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59999829-a593-49f1-a77c-ca128f16e2c6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:19.433409 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 175/353] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:19.434371 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59999829-a593-49f1-a77c-ca128f16e2c6 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:19.516715 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c05f1-27d3-4bd7-b7aa-0ac04aa3b7fa None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:19.541578 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59999829-a593-49f1-a77c-ca128f16e2c6 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:19.556783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cff380-e6f5-4129-9cfc-6c21c1558c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:19.557966 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cff380-e6f5-4129-9cfc-6c21c1558c47 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:19.558518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cff380-e6f5-4129-9cfc-6c21c1558c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:19.558992 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cff380-e6f5-4129-9cfc-6c21c1558c47 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:19.571203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cff380-e6f5-4129-9cfc-6c21c1558c47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:19.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e7e33-3796-4de1-b97a-3aab8e51f0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:19.580041 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e7e33-3796-4de1-b97a-3aab8e51f0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:19.580332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e7e33-3796-4de1-b97a-3aab8e51f0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-94cff380-e6f5-4129-9cfc-6c21c1558c47 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:19.580435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:19.581316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:19.582169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:19.582169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:19.582169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:19.582169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:19.582169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:19.582169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:19.582169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:19.582169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:19.582169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:19.582169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:19.582169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:19.582169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:19.582788 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e7e33-3796-4de1-b97a-3aab8e51f0a9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:19.584665 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 179/354] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:21:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:19.585748 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cff380-e6f5-4129-9cfc-6c21c1558c47 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:19.599149 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e7e33-3796-4de1-b97a-3aab8e51f0a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-e58e7e33-3796-4de1-b97a-3aab8e51f0a9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:19.607450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:19.608454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:19.609368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:19.609368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:19.609368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:19.609368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:19.609368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:19.609368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:19.609368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:19.609368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:19.609368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:19.609368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:19.609368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:21:19.609368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:19.609368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 176/355] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:19.610588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e7e33-3796-4de1-b97a-3aab8e51f0a9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:19.628611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:19.634692 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:19.710741 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cff380-e6f5-4129-9cfc-6c21c1558c47 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:19.725710 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e7e33-3796-4de1-b97a-3aab8e51f0a9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:19.746758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17b608-b2c9-4ab4-99f1-c36a4c2088e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:19.747436 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17b608-b2c9-4ab4-99f1-c36a4c2088e6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:19.747722 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17b608-b2c9-4ab4-99f1-c36a4c2088e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:19.747999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17b608-b2c9-4ab4-99f1-c36a4c2088e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:19.748746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17b608-b2c9-4ab4-99f1-c36a4c2088e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:19.749072 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17b608-b2c9-4ab4-99f1-c36a4c2088e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:19.750484 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17b608-b2c9-4ab4-99f1-c36a4c2088e6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:19.760168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 180/356] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:19.760842 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17b608-b2c9-4ab4-99f1-c36a4c2088e6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:19.763984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae007e9-1891-42b2-84c6-023a26339528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:19.764657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae007e9-1891-42b2-84c6-023a26339528 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:19.764951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae007e9-1891-42b2-84c6-023a26339528 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:19.765264 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae007e9-1891-42b2-84c6-023a26339528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:19.841507 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae007e9-1891-42b2-84c6-023a26339528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:19.843003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae007e9-1891-42b2-84c6-023a26339528 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:19.860631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:19.883461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17b608-b2c9-4ab4-99f1-c36a4c2088e6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:19.885591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:19.927143 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2f096-0db0-49ba-8f5d-275f59e7cac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:19.927826 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2f096-0db0-49ba-8f5d-275f59e7cac8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:19.928374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2f096-0db0-49ba-8f5d-275f59e7cac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:19.928754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2f096-0db0-49ba-8f5d-275f59e7cac8 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:19.929884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2f096-0db0-49ba-8f5d-275f59e7cac8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:19.930201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2f096-0db0-49ba-8f5d-275f59e7cac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:19.931568 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2f096-0db0-49ba-8f5d-275f59e7cac8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:19.940749 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 181/357] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:19.941433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2f096-0db0-49ba-8f5d-275f59e7cac8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:19.989016 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 177/358] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:19.990046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae007e9-1891-42b2-84c6-023a26339528 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:20.074398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2f096-0db0-49ba-8f5d-275f59e7cac8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:20.115806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae007e9-1891-42b2-84c6-023a26339528 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:20.118551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd7dac-2cb3-40be-aea8-ae919364ce3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:20.119226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd7dac-2cb3-40be-aea8-ae919364ce3a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:20.119511 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd7dac-2cb3-40be-aea8-ae919364ce3a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:20.119785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd7dac-2cb3-40be-aea8-ae919364ce3a None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:20.120529 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd7dac-2cb3-40be-aea8-ae919364ce3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:20.120836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd7dac-2cb3-40be-aea8-ae919364ce3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:20.122248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd7dac-2cb3-40be-aea8-ae919364ce3a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:20.131675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:20.131879 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 182/359] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:20.138176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab6251-ef76-4b14-9771-5c88c54ef1ce None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:20.138176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab6251-ef76-4b14-9771-5c88c54ef1ce None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:20.138176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab6251-ef76-4b14-9771-5c88c54ef1ce None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:20.139723 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 183/360] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:20.140566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab6251-ef76-4b14-9771-5c88c54ef1ce None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:20.160073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d87e3a-6294-44a0-979c-9c48d320faa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:20.160754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d87e3a-6294-44a0-979c-9c48d320faa1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:20.161049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d87e3a-6294-44a0-979c-9c48d320faa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:20.161422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d87e3a-6294-44a0-979c-9c48d320faa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:20.175706 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:20.256924 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d87e3a-6294-44a0-979c-9c48d320faa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:20.257736 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d87e3a-6294-44a0-979c-9c48d320faa1 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:20.275832 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab6251-ef76-4b14-9771-5c88c54ef1ce None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:20.328680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e6bc6-188e-44a9-b768-904db9ed3f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:20.329382 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e6bc6-188e-44a9-b768-904db9ed3f8f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:20.329773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e6bc6-188e-44a9-b768-904db9ed3f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:20.330054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e6bc6-188e-44a9-b768-904db9ed3f8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:20.387038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 178/361] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:20.388176 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d87e3a-6294-44a0-979c-9c48d320faa1 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:20.406658 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:20.424078 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e6bc6-188e-44a9-b768-904db9ed3f8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:20.425910 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e6bc6-188e-44a9-b768-904db9ed3f8f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:20.462562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:20.517516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d87e3a-6294-44a0-979c-9c48d320faa1 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:20.558766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada243c2-b03a-47e5-bae4-170e5e02954e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:20.559637 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada243c2-b03a-47e5-bae4-170e5e02954e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:20.559953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada243c2-b03a-47e5-bae4-170e5e02954e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:20.560200 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada243c2-b03a-47e5-bae4-170e5e02954e None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:20.561430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada243c2-b03a-47e5-bae4-170e5e02954e None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:20.562885 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada243c2-b03a-47e5-bae4-170e5e02954e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:20.563356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 184/362] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:20.564639 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 179/363] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:20 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:20.564946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e6bc6-188e-44a9-b768-904db9ed3f8f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:20.565887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e3068-8e07-459f-9ac5-6c835684ec5e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:20.566173 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e3068-8e07-459f-9ac5-6c835684ec5e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:20.566443 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e3068-8e07-459f-9ac5-6c835684ec5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:20.585881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-430e3068-8e07-459f-9ac5-6c835684ec5e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:20.588453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-430e3068-8e07-459f-9ac5-6c835684ec5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:20.675873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 180/364] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:20.676666 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e3068-8e07-459f-9ac5-6c835684ec5e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:20.695346 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:20.707002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e6bc6-188e-44a9-b768-904db9ed3f8f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:20.734637 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:20.751015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f6b55-67b9-44c4-b4c9-584e06c4685d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:20.752050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f6b55-67b9-44c4-b4c9-584e06c4685d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:20.752332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f6b55-67b9-44c4-b4c9-584e06c4685d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:20.752606 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f6b55-67b9-44c4-b4c9-584e06c4685d None admin] PATH_INFO: `/v3/users/7d5f6895ba4b40d9a278b39d7ffbe782` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:20.753906 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f6b55-67b9-44c4-b4c9-584e06c4685d None admin] RBAC: Authorizing `identity:get_user(user_id=7d5f6895ba4b40d9a278b39d7ffbe782)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:20.755744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f6b55-67b9-44c4-b4c9-584e06c4685d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:20.758114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 185/365] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:21:20 2022] GET /identity/v3/users/7d5f6895ba4b40d9a278b39d7ffbe782 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:20.758910 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f6b55-67b9-44c4-b4c9-584e06c4685d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:20.812208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e3068-8e07-459f-9ac5-6c835684ec5e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:20.859761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ef757-ea47-488a-9116-a56b71884d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:20.860630 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ef757-ea47-488a-9116-a56b71884d87 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:20.860914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ef757-ea47-488a-9116-a56b71884d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:20.861225 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ef757-ea47-488a-9116-a56b71884d87 None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:20.862407 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ef757-ea47-488a-9116-a56b71884d87 None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:20.863982 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ef757-ea47-488a-9116-a56b71884d87 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:20.868373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 181/366] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:20 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:20.868373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ef757-ea47-488a-9116-a56b71884d87 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:20.902701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f6b55-67b9-44c4-b4c9-584e06c4685d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:20.951063 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802b063-8356-42b0-94f2-50e2e0bfc025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:20.951853 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802b063-8356-42b0-94f2-50e2e0bfc025 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:20.952136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802b063-8356-42b0-94f2-50e2e0bfc025 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:20.952410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802b063-8356-42b0-94f2-50e2e0bfc025 None admin] PATH_INFO: `/v3/groups/a83da4807fb249adb95a024797878b3b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:20.953615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802b063-8356-42b0-94f2-50e2e0bfc025 None admin] RBAC: Authorizing `identity:get_group(group_id=a83da4807fb249adb95a024797878b3b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:20.955021 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802b063-8356-42b0-94f2-50e2e0bfc025 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:20.956647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 186/367] 10.222.0.64 () {56 vars in 1180 bytes} [Mon May 23 15:21:20 2022] GET /identity/v3/groups/a83da4807fb249adb95a024797878b3b => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:20.957963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802b063-8356-42b0-94f2-50e2e0bfc025 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:20.979884 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:20.999211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ef757-ea47-488a-9116-a56b71884d87 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:21.030488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:21.045618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ee0d1-180a-445a-a755-18408a5e0378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:21.046667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ee0d1-180a-445a-a755-18408a5e0378 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:21.046959 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ee0d1-180a-445a-a755-18408a5e0378 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:21.047241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ee0d1-180a-445a-a755-18408a5e0378 None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:21.049410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ee0d1-180a-445a-a755-18408a5e0378 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:21.050883 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ee0d1-180a-445a-a755-18408a5e0378 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:21.053737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 182/368] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:20 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:21.054378 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ee0d1-180a-445a-a755-18408a5e0378 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:21.083171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802b063-8356-42b0-94f2-50e2e0bfc025 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:21.123616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0c185-2f33-4138-a55c-327f4b4118f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:21.124434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0c185-2f33-4138-a55c-327f4b4118f3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:21.124717 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0c185-2f33-4138-a55c-327f4b4118f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:21.124992 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0c185-2f33-4138-a55c-327f4b4118f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:21.140275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed0c185-2f33-4138-a55c-327f4b4118f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-5ed0c185-2f33-4138-a55c-327f4b4118f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:21.149604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:21.150599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:21.151405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:21.151405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:21.151405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:21.151405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:21.151405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:21.151405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:21.151405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:21.151405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:21.151405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:21.151405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:21.151405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:21:21.151405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:21.151405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 187/369] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:21:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:21.152103 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0c185-2f33-4138-a55c-327f4b4118f3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:21.189015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ee0d1-180a-445a-a755-18408a5e0378 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:21.233876 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dba7e5-a2fa-434d-97cb-9a51825b7313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:21.234751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dba7e5-a2fa-434d-97cb-9a51825b7313 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:21.235105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dba7e5-a2fa-434d-97cb-9a51825b7313 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:21.235394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dba7e5-a2fa-434d-97cb-9a51825b7313 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:21.236747 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dba7e5-a2fa-434d-97cb-9a51825b7313 None admin] RBAC: Adding query filter params (role.id=1026857a0bd64410a463448b53532814, user.id=7d5f6895ba4b40d9a278b39d7ffbe782, scope.system=all) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:21.237193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dba7e5-a2fa-434d-97cb-9a51825b7313 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:21.238554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dba7e5-a2fa-434d-97cb-9a51825b7313 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:21.252939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 183/370] 10.222.0.64 () {56 vars in 1308 bytes} [Mon May 23 15:21:21 2022] GET /identity/v3/role_assignments?user.id=7d5f6895ba4b40d9a278b39d7ffbe782&scope.system=all&role.id=1026857a0bd64410a463448b53532814 => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:21.254236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dba7e5-a2fa-434d-97cb-9a51825b7313 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:21.269181 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:21.293191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0c185-2f33-4138-a55c-327f4b4118f3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:21.318466 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:21.343070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a17d7-ebd3-4932-a4d6-f31813914c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:21.344084 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a17d7-ebd3-4932-a4d6-f31813914c5c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:21.344322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a17d7-ebd3-4932-a4d6-f31813914c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:21.344624 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a17d7-ebd3-4932-a4d6-f31813914c5c None admin] PATH_INFO: `/v3/groups/a83da4807fb249adb95a024797878b3b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:21.346180 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a17d7-ebd3-4932-a4d6-f31813914c5c None admin] RBAC: Authorizing `identity:get_group(group_id=a83da4807fb249adb95a024797878b3b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:21.351416 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a17d7-ebd3-4932-a4d6-f31813914c5c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:21.351416 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 188/371] 10.222.0.64 () {56 vars in 1180 bytes} [Mon May 23 15:21:21 2022] GET /identity/v3/groups/a83da4807fb249adb95a024797878b3b => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:21.352679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a17d7-ebd3-4932-a4d6-f31813914c5c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:21.405007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dba7e5-a2fa-434d-97cb-9a51825b7313 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:21.466253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c24904-1a4f-48be-8d66-9ed6e557667e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:21.466961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c24904-1a4f-48be-8d66-9ed6e557667e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:21.467254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c24904-1a4f-48be-8d66-9ed6e557667e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:21.467844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c24904-1a4f-48be-8d66-9ed6e557667e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:21.468733 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c24904-1a4f-48be-8d66-9ed6e557667e None admin] RBAC: Adding query filter params (group.id=a83da4807fb249adb95a024797878b3b, role.id=9330473f62ec4a82a82d1c4beb6160d7, scope.project.id=bc8a10b282fb412fbc79062901f84a10) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:21.469037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c24904-1a4f-48be-8d66-9ed6e557667e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:21.471328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c24904-1a4f-48be-8d66-9ed6e557667e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:21.477572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 184/372] 10.222.0.64 () {56 vars in 1376 bytes} [Mon May 23 15:21:21 2022] GET /identity/v3/role_assignments?group.id=a83da4807fb249adb95a024797878b3b&scope.project.id=bc8a10b282fb412fbc79062901f84a10&role.id=9330473f62ec4a82a82d1c4beb6160d7 => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:21.478618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c24904-1a4f-48be-8d66-9ed6e557667e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:21.498639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a17d7-ebd3-4932-a4d6-f31813914c5c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:21.553721 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a921699-7f12-46dc-8dc3-8f830e9289f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:21.554651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a921699-7f12-46dc-8dc3-8f830e9289f3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:21.554938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a921699-7f12-46dc-8dc3-8f830e9289f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:21.555216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a921699-7f12-46dc-8dc3-8f830e9289f3 None admin] PATH_INFO: `/v3/groups/a83da4807fb249adb95a024797878b3b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:21.556536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a921699-7f12-46dc-8dc3-8f830e9289f3 None admin] RBAC: Authorizing `identity:get_group(group_id=a83da4807fb249adb95a024797878b3b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:21.558037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a921699-7f12-46dc-8dc3-8f830e9289f3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:21.559953 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 189/373] 10.222.0.64 () {56 vars in 1180 bytes} [Mon May 23 15:21:21 2022] GET /identity/v3/groups/a83da4807fb249adb95a024797878b3b => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:21.560591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a921699-7f12-46dc-8dc3-8f830e9289f3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:21.583561 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:21.628651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c24904-1a4f-48be-8d66-9ed6e557667e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:21.634722 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:21.678679 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b60fb5-0762-4d7a-9d14-7df44ecf5eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:21.679847 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b60fb5-0762-4d7a-9d14-7df44ecf5eef None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:21.679847 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b60fb5-0762-4d7a-9d14-7df44ecf5eef None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:21.680023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b60fb5-0762-4d7a-9d14-7df44ecf5eef None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:21.680712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b60fb5-0762-4d7a-9d14-7df44ecf5eef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:21.681027 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b60fb5-0762-4d7a-9d14-7df44ecf5eef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:21.684572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b60fb5-0762-4d7a-9d14-7df44ecf5eef None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:21.694031 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 185/374] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:21.695127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb4175-df70-4066-b11c-97bae3e97663 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:21.695682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb4175-df70-4066-b11c-97bae3e97663 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:21.695682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb4175-df70-4066-b11c-97bae3e97663 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:21.697281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 186/375] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:21.702805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb4175-df70-4066-b11c-97bae3e97663 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:21.714693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a921699-7f12-46dc-8dc3-8f830e9289f3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:21.758278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb056a2-0656-4bc9-8035-f9e279464d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:21.759463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb056a2-0656-4bc9-8035-f9e279464d74 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:21.759463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb056a2-0656-4bc9-8035-f9e279464d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:21.759763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb056a2-0656-4bc9-8035-f9e279464d74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:21.760727 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb056a2-0656-4bc9-8035-f9e279464d74 None admin] RBAC: Adding query filter params (group.id=a83da4807fb249adb95a024797878b3b, role.id=1bac0428b2544e2cb16fad539f080c14, scope.project.id=936684d8d665414698e461a87200ef9b) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:21.761168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb056a2-0656-4bc9-8035-f9e279464d74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:21.762708 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb056a2-0656-4bc9-8035-f9e279464d74 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:21.768761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 190/376] 10.222.0.64 () {56 vars in 1376 bytes} [Mon May 23 15:21:21 2022] GET /identity/v3/role_assignments?group.id=a83da4807fb249adb95a024797878b3b&scope.project.id=936684d8d665414698e461a87200ef9b&role.id=1bac0428b2544e2cb16fad539f080c14 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:21.769654 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb056a2-0656-4bc9-8035-f9e279464d74 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:21.827896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb4175-df70-4066-b11c-97bae3e97663 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:21.867458 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:21.872378 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963f654-e749-4ace-8896-8772f3f90c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:21.873459 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963f654-e749-4ace-8896-8772f3f90c84 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:21.873958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963f654-e749-4ace-8896-8772f3f90c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:21.874445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963f654-e749-4ace-8896-8772f3f90c84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:21.875622 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963f654-e749-4ace-8896-8772f3f90c84 None admin] RBAC: Adding query filter params (group.id=a83da4807fb249adb95a024797878b3b, role.id=1bac0428b2544e2cb16fad539f080c14, scope.project.id=bc8a10b282fb412fbc79062901f84a10) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:21.876236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963f654-e749-4ace-8896-8772f3f90c84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:21.877992 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963f654-e749-4ace-8896-8772f3f90c84 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:21.885209 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 187/377] 10.222.0.64 () {56 vars in 1376 bytes} [Mon May 23 15:21:21 2022] GET /identity/v3/role_assignments?group.id=a83da4807fb249adb95a024797878b3b&scope.project.id=bc8a10b282fb412fbc79062901f84a10&role.id=1bac0428b2544e2cb16fad539f080c14 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:21.886837 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecdb4f-b209-4f86-8ea7-2ed8d69ba03f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:21.893267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecdb4f-b209-4f86-8ea7-2ed8d69ba03f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:21.894030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecdb4f-b209-4f86-8ea7-2ed8d69ba03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:21.900586 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb056a2-0656-4bc9-8035-f9e279464d74 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:21.919252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-93ecdb4f-b209-4f86-8ea7-2ed8d69ba03f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:21.920427 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:21.921005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-93ecdb4f-b209-4f86-8ea7-2ed8d69ba03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:21.940107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd8331-7930-4307-8466-e52128b729c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:21.940784 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd8331-7930-4307-8466-e52128b729c9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:21.941089 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd8331-7930-4307-8466-e52128b729c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:21.941386 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd8331-7930-4307-8466-e52128b729c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:22.031690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd8331-7930-4307-8466-e52128b729c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:22.033343 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd8331-7930-4307-8466-e52128b729c9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:22.085496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 188/378] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:22.119263 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecdb4f-b209-4f86-8ea7-2ed8d69ba03f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:22.220004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 191/379] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:22.230343 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd8331-7930-4307-8466-e52128b729c9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:22.249429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecdb4f-b209-4f86-8ea7-2ed8d69ba03f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:22.282821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:22.285506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:22.289018 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676631c2-f2d7-4a91-bfe6-55fe1eafc442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:22.289859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676631c2-f2d7-4a91-bfe6-55fe1eafc442 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:22.290147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676631c2-f2d7-4a91-bfe6-55fe1eafc442 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:22.290428 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676631c2-f2d7-4a91-bfe6-55fe1eafc442 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:22.303500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676631c2-f2d7-4a91-bfe6-55fe1eafc442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-676631c2-f2d7-4a91-bfe6-55fe1eafc442 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:22.311463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:22.312359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:22.313224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:22.313224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:22.313224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:22.313224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:22.313224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:22.313224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:22.313224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:22.313224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:22.313224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:22.313224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:22.313224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:22.313224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:22.313224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 189/380] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:21:22.319578 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676631c2-f2d7-4a91-bfe6-55fe1eafc442 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:22.431949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd8331-7930-4307-8466-e52128b729c9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:22.455620 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676631c2-f2d7-4a91-bfe6-55fe1eafc442 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:22.473657 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51691318-907a-437b-89c1-30d25fdb0afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:22.475166 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51691318-907a-437b-89c1-30d25fdb0afd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:22.475454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51691318-907a-437b-89c1-30d25fdb0afd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:22.475731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51691318-907a-437b-89c1-30d25fdb0afd None admin] PATH_INFO: `/v3/projects/4c9cfa8acbce47f6b0259cbfbf5b338d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:22.477155 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51691318-907a-437b-89c1-30d25fdb0afd None admin] RBAC: Authorizing `identity:get_project(project_id=4c9cfa8acbce47f6b0259cbfbf5b338d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:22.478613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51691318-907a-437b-89c1-30d25fdb0afd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:22.482945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 192/381] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:22 2022] GET /identity/v3/projects/4c9cfa8acbce47f6b0259cbfbf5b338d => generated 344 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:22.490808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51691318-907a-437b-89c1-30d25fdb0afd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:22.531990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7fa06-a1d9-4768-ac03-e14c03734d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:22.533177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7fa06-a1d9-4768-ac03-e14c03734d84 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:22.533665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7fa06-a1d9-4768-ac03-e14c03734d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:22.534138 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7fa06-a1d9-4768-ac03-e14c03734d84 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:22.535113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7fa06-a1d9-4768-ac03-e14c03734d84 None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:22.535616 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7fa06-a1d9-4768-ac03-e14c03734d84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:22.537395 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7fa06-a1d9-4768-ac03-e14c03734d84 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:22.553906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 190/382] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:21:22.559940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7fa06-a1d9-4768-ac03-e14c03734d84 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:22.623559 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:22.629388 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:22.671248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51691318-907a-437b-89c1-30d25fdb0afd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:22.683251 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7fa06-a1d9-4768-ac03-e14c03734d84 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:22.719738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bad65-89a8-45fa-8fa9-ab03b18d4c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:22.720617 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bad65-89a8-45fa-8fa9-ab03b18d4c94 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:22.720896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bad65-89a8-45fa-8fa9-ab03b18d4c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:22.721220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bad65-89a8-45fa-8fa9-ab03b18d4c94 None admin] PATH_INFO: `/v3/groups/a83da4807fb249adb95a024797878b3b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:22.722427 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bad65-89a8-45fa-8fa9-ab03b18d4c94 None admin] RBAC: Authorizing `identity:get_group(group_id=a83da4807fb249adb95a024797878b3b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:22.724425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bad65-89a8-45fa-8fa9-ab03b18d4c94 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:22.726537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 191/383] 10.222.0.64 () {56 vars in 1180 bytes} [Mon May 23 15:21:22 2022] GET /identity/v3/groups/a83da4807fb249adb95a024797878b3b => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:21:22.731695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bad65-89a8-45fa-8fa9-ab03b18d4c94 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:22.749809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90916e51-69e9-4e75-be3a-6f2b192d4c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:22.751028 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90916e51-69e9-4e75-be3a-6f2b192d4c1f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:22.751541 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90916e51-69e9-4e75-be3a-6f2b192d4c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:22.752043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90916e51-69e9-4e75-be3a-6f2b192d4c1f None admin] PATH_INFO: `/v3/groups/7869dc6bbe714697bfd76b940253cf6a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:22.757595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90916e51-69e9-4e75-be3a-6f2b192d4c1f None admin] RBAC: Authorizing `identity:get_group(group_id=7869dc6bbe714697bfd76b940253cf6a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:22.765081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90916e51-69e9-4e75-be3a-6f2b192d4c1f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:22.767408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 193/384] 10.222.0.64 () {56 vars in 1180 bytes} [Mon May 23 15:21:22 2022] GET /identity/v3/groups/7869dc6bbe714697bfd76b940253cf6a => generated 229 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:22.775071 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90916e51-69e9-4e75-be3a-6f2b192d4c1f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:22.846601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bad65-89a8-45fa-8fa9-ab03b18d4c94 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:22.863760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:22.883360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df22623-7899-4ad3-b993-1603d8e34b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:22.884233 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df22623-7899-4ad3-b993-1603d8e34b60 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:22.884531 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df22623-7899-4ad3-b993-1603d8e34b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:22.884807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df22623-7899-4ad3-b993-1603d8e34b60 None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:22.886049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df22623-7899-4ad3-b993-1603d8e34b60 None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:22.887508 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df22623-7899-4ad3-b993-1603d8e34b60 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:22.889799 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 192/385] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:22 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:21:22.896526 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df22623-7899-4ad3-b993-1603d8e34b60 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:23.016788 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90916e51-69e9-4e75-be3a-6f2b192d4c1f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:23.026005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df22623-7899-4ad3-b993-1603d8e34b60 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:23.062870 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410ee30-4e0e-4167-992d-e0bf3e844bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:23.063770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410ee30-4e0e-4167-992d-e0bf3e844bcd None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:23.064064 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410ee30-4e0e-4167-992d-e0bf3e844bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:23.064349 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410ee30-4e0e-4167-992d-e0bf3e844bcd None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b/groups/a83da4807fb249adb95a024797878b3b/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:23.066596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410ee30-4e0e-4167-992d-e0bf3e844bcd None admin] RBAC: Authorizing `identity:create_grant(project_id=936684d8d665414698e461a87200ef9b, group_id=a83da4807fb249adb95a024797878b3b, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:23.068133 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410ee30-4e0e-4167-992d-e0bf3e844bcd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:23.092311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d8e36-73e0-42d1-a997-83d4a0bdbd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:23.093194 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d8e36-73e0-42d1-a997-83d4a0bdbd1d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:23.093536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d8e36-73e0-42d1-a997-83d4a0bdbd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:23.093827 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d8e36-73e0-42d1-a997-83d4a0bdbd1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:23.094735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d8e36-73e0-42d1-a997-83d4a0bdbd1d None admin] RBAC: Adding query filter params (group.id=7869dc6bbe714697bfd76b940253cf6a, role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=4c9cfa8acbce47f6b0259cbfbf5b338d) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:23.095056 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d8e36-73e0-42d1-a997-83d4a0bdbd1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:23.096656 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d8e36-73e0-42d1-a997-83d4a0bdbd1d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:23.103506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 194/386] 10.222.0.64 () {56 vars in 1376 bytes} [Mon May 23 15:21:22 2022] GET /identity/v3/role_assignments?group.id=7869dc6bbe714697bfd76b940253cf6a&scope.project.id=4c9cfa8acbce47f6b0259cbfbf5b338d&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 260 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:23.163433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 193/387] 10.222.0.64 () {58 vars in 1442 bytes} [Mon May 23 15:21:22 2022] PUT /identity/v3/projects/936684d8d665414698e461a87200ef9b/groups/a83da4807fb249adb95a024797878b3b/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:21:24.445311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bf49f-7e2e-4aa2-ab1f-749150898ee6 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:24.445831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bf49f-7e2e-4aa2-ab1f-749150898ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:24.445939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bf49f-7e2e-4aa2-ab1f-749150898ee6 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:24.447500 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 195/388] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:24.452728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09262101-282d-4c78-a420-8687c65af69f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:24.453178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09262101-282d-4c78-a420-8687c65af69f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:24.453354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09262101-282d-4c78-a420-8687c65af69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:24.455971 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619b7ff-08e2-4714-9ccf-0b3bc1736680 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:24.456261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619b7ff-08e2-4714-9ccf-0b3bc1736680 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:24.456538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619b7ff-08e2-4714-9ccf-0b3bc1736680 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:24.457827 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 196/389] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:24.464038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8df2e-c3f2-47c6-88ec-3b4735701c79 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:24.464326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8df2e-c3f2-47c6-88ec-3b4735701c79 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:24.464608 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8df2e-c3f2-47c6-88ec-3b4735701c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:24.481206 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-09262101-282d-4c78-a420-8687c65af69f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:24.482401 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-09262101-282d-4c78-a420-8687c65af69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:24.499215 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a3d8df2e-c3f2-47c6-88ec-3b4735701c79 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:24.500394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a3d8df2e-c3f2-47c6-88ec-3b4735701c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:24.512116 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:24.547473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:24.584991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 197/391] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:24.584991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0ddac-8ea4-4d5a-845d-c1e48be8d699 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:24.584991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0ddac-8ea4-4d5a-845d-c1e48be8d699 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:24.584991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0ddac-8ea4-4d5a-845d-c1e48be8d699 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:24.584991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 198/392] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:24.585460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 194/392] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:21:24.590374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877d358-18a6-4c4e-9dc4-2dcae8d5d630 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:24.591912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877d358-18a6-4c4e-9dc4-2dcae8d5d630 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:24.591912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877d358-18a6-4c4e-9dc4-2dcae8d5d630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:24.603325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09262101-282d-4c78-a420-8687c65af69f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:24.610310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a877d358-18a6-4c4e-9dc4-2dcae8d5d630 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:24.611541 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a877d358-18a6-4c4e-9dc4-2dcae8d5d630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:24.686577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 199/393] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:24.687428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877d358-18a6-4c4e-9dc4-2dcae8d5d630 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:24.746257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09262101-282d-4c78-a420-8687c65af69f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:24.795645 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ac11b-2aae-4497-b09c-47dee05fc2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:24.797340 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ac11b-2aae-4497-b09c-47dee05fc2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:24.797676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ac11b-2aae-4497-b09c-47dee05fc2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:24.797960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ac11b-2aae-4497-b09c-47dee05fc2d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:24.801573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:24.814432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ac11b-2aae-4497-b09c-47dee05fc2d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-f78ac11b-2aae-4497-b09c-47dee05fc2d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:24.823599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:24.825693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:24.826498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:24.826498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:24.826498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:24.826498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:24.826498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:24.826498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:24.826498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:24.826498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:24.826498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:24.826498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:24.826498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:24.826498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:24.826498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 195/394] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:21:24.826498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ac11b-2aae-4497-b09c-47dee05fc2d8 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:24.836491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877d358-18a6-4c4e-9dc4-2dcae8d5d630 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:24.863095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:24.887028 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083783bf-911a-49fd-b94a-15484478e96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:24.888553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083783bf-911a-49fd-b94a-15484478e96b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:24.889158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083783bf-911a-49fd-b94a-15484478e96b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:24.889679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083783bf-911a-49fd-b94a-15484478e96b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:24.904965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083783bf-911a-49fd-b94a-15484478e96b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-083783bf-911a-49fd-b94a-15484478e96b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:24.917374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:24.918489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:24.919490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:24.919490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:24.919490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:24.919490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:24.919490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:24.919490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:24.919490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:24.919490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:24.919490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:24.919490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:24.919490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:24.919490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:24.922182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 200/395] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:21:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:24.923111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083783bf-911a-49fd-b94a-15484478e96b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:24.943811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ac11b-2aae-4497-b09c-47dee05fc2d8 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:24.979252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be146f9-72c6-4734-aa65-7ff688114305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:24.980165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be146f9-72c6-4734-aa65-7ff688114305 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:24.980539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be146f9-72c6-4734-aa65-7ff688114305 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:24.980910 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be146f9-72c6-4734-aa65-7ff688114305 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.003145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be146f9-72c6-4734-aa65-7ff688114305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:25.003145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-7be146f9-72c6-4734-aa65-7ff688114305 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:25.003145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:25.003145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:25.003145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:25.003145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:25.003145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:25.003145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:25.003145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:25.003145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:25.003145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:25.003145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:25.003145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:25.003918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:25.004769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:25.004769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:25.004769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:25.004769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:25.004769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:25.004769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:25.004769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:25.004769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:25.004769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:25.004769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:25.004769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:25.004769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:25.004769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:25.004769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:25.004769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:25.008695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 196/396] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:21:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:25.009652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be146f9-72c6-4734-aa65-7ff688114305 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:25.065123 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083783bf-911a-49fd-b94a-15484478e96b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:25.105315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce85968-0728-4cf6-b73f-3b677f76f686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:25.106285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce85968-0728-4cf6-b73f-3b677f76f686 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:25.106778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce85968-0728-4cf6-b73f-3b677f76f686 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:25.107234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce85968-0728-4cf6-b73f-3b677f76f686 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.108266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce85968-0728-4cf6-b73f-3b677f76f686 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:25.108753 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce85968-0728-4cf6-b73f-3b677f76f686 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:25.110307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce85968-0728-4cf6-b73f-3b677f76f686 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:25.112067 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:25.113902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:25.121408 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be146f9-72c6-4734-aa65-7ff688114305 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:25.122890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 201/397] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:25.124134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce85968-0728-4cf6-b73f-3b677f76f686 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:25.160032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c8888-c295-4d89-99c1-82e3c466e33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:25.160726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c8888-c295-4d89-99c1-82e3c466e33b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:25.161000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c8888-c295-4d89-99c1-82e3c466e33b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:25.162314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c8888-c295-4d89-99c1-82e3c466e33b None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.163649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c8888-c295-4d89-99c1-82e3c466e33b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:25.164834 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c8888-c295-4d89-99c1-82e3c466e33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:25.165704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c8888-c295-4d89-99c1-82e3c466e33b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:25.175008 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 197/398] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:25.175833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c8888-c295-4d89-99c1-82e3c466e33b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:25.238750 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce85968-0728-4cf6-b73f-3b677f76f686 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:25.276930 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b49da-8ee4-437a-9b09-0753e93802cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:25.278133 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b49da-8ee4-437a-9b09-0753e93802cb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:25.278616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b49da-8ee4-437a-9b09-0753e93802cb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:25.279088 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b49da-8ee4-437a-9b09-0753e93802cb None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.280031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b49da-8ee4-437a-9b09-0753e93802cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:25.280640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b49da-8ee4-437a-9b09-0753e93802cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:25.282394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b49da-8ee4-437a-9b09-0753e93802cb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:25.290896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c8888-c295-4d89-99c1-82e3c466e33b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:25.292465 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 202/399] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:25.293659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b49da-8ee4-437a-9b09-0753e93802cb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:25.332417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d285c36-832a-4ae1-bea9-19ac4d34ccd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:25.333935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d285c36-832a-4ae1-bea9-19ac4d34ccd7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:25.334393 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d285c36-832a-4ae1-bea9-19ac4d34ccd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:25.334677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d285c36-832a-4ae1-bea9-19ac4d34ccd7 None admin] PATH_INFO: `/v3/groups/a83da4807fb249adb95a024797878b3b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.335893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d285c36-832a-4ae1-bea9-19ac4d34ccd7 None admin] RBAC: Authorizing `identity:get_group(group_id=a83da4807fb249adb95a024797878b3b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:25.337391 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d285c36-832a-4ae1-bea9-19ac4d34ccd7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:25.339159 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 198/400] 10.222.0.64 () {56 vars in 1180 bytes} [Mon May 23 15:21:25 2022] GET /identity/v3/groups/a83da4807fb249adb95a024797878b3b => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:25.339940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d285c36-832a-4ae1-bea9-19ac4d34ccd7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:25.358819 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:25.370567 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:25.408639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b49da-8ee4-437a-9b09-0753e93802cb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:25.447174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e1b81-c4da-4820-84db-a21897a5e8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:25.448050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e1b81-c4da-4820-84db-a21897a5e8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:25.448335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e1b81-c4da-4820-84db-a21897a5e8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:25.448615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e1b81-c4da-4820-84db-a21897a5e8d3 None admin] PATH_INFO: `/v3/groups/a83da4807fb249adb95a024797878b3b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.450287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e1b81-c4da-4820-84db-a21897a5e8d3 None admin] RBAC: Authorizing `identity:get_group(group_id=a83da4807fb249adb95a024797878b3b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:25.451541 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e1b81-c4da-4820-84db-a21897a5e8d3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:25.453628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 203/401] 10.222.0.64 () {56 vars in 1180 bytes} [Mon May 23 15:21:25 2022] GET /identity/v3/groups/a83da4807fb249adb95a024797878b3b => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:25.454663 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e1b81-c4da-4820-84db-a21897a5e8d3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:25.475624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d285c36-832a-4ae1-bea9-19ac4d34ccd7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:25.522937 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f91a6-4d59-47f3-97f3-4594f2aae62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:25.523719 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f91a6-4d59-47f3-97f3-4594f2aae62d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:25.524003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f91a6-4d59-47f3-97f3-4594f2aae62d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:25.524276 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f91a6-4d59-47f3-97f3-4594f2aae62d None admin] PATH_INFO: `/v3/groups/a83da4807fb249adb95a024797878b3b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.525606 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f91a6-4d59-47f3-97f3-4594f2aae62d None admin] RBAC: Authorizing `identity:get_group(group_id=a83da4807fb249adb95a024797878b3b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:25.527042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f91a6-4d59-47f3-97f3-4594f2aae62d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:25.533298 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 199/402] 10.222.0.64 () {56 vars in 1180 bytes} [Mon May 23 15:21:25 2022] GET /identity/v3/groups/a83da4807fb249adb95a024797878b3b => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:25.533298 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f91a6-4d59-47f3-97f3-4594f2aae62d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:25.592029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e1b81-c4da-4820-84db-a21897a5e8d3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:25.629249 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:25.641797 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090c628-9603-45b3-967e-d6cdd1db5378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:25.642751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:25.644460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090c628-9603-45b3-967e-d6cdd1db5378 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:25.645151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090c628-9603-45b3-967e-d6cdd1db5378 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:25.645805 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090c628-9603-45b3-967e-d6cdd1db5378 None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.647575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090c628-9603-45b3-967e-d6cdd1db5378 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:25.649025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090c628-9603-45b3-967e-d6cdd1db5378 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:25.655545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 204/403] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:25 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:25.655545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090c628-9603-45b3-967e-d6cdd1db5378 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:25.662391 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f91a6-4d59-47f3-97f3-4594f2aae62d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:25.709101 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4720c37-89c0-41d0-9d88-26a3ac30dfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:25.710090 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4720c37-89c0-41d0-9d88-26a3ac30dfeb None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:25.710383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4720c37-89c0-41d0-9d88-26a3ac30dfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:25.710700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4720c37-89c0-41d0-9d88-26a3ac30dfeb None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.712055 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4720c37-89c0-41d0-9d88-26a3ac30dfeb None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:25.713692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4720c37-89c0-41d0-9d88-26a3ac30dfeb None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:25.715885 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 200/404] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:25 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:25.716836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4720c37-89c0-41d0-9d88-26a3ac30dfeb None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:25.785092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090c628-9603-45b3-967e-d6cdd1db5378 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:25.822120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1509e9b-cb54-450a-a7a0-12c1a5609f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:25.822981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1509e9b-cb54-450a-a7a0-12c1a5609f7b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:25.823268 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1509e9b-cb54-450a-a7a0-12c1a5609f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:25.823546 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1509e9b-cb54-450a-a7a0-12c1a5609f7b None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.824720 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1509e9b-cb54-450a-a7a0-12c1a5609f7b None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:25.826175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1509e9b-cb54-450a-a7a0-12c1a5609f7b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:25.828020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 205/405] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:25 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:25.828661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1509e9b-cb54-450a-a7a0-12c1a5609f7b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:25.839096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4720c37-89c0-41d0-9d88-26a3ac30dfeb None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:25.876140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b767b-eae9-4f52-a204-441f0756bd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:25.877339 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b767b-eae9-4f52-a204-441f0756bd84 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:25.877944 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b767b-eae9-4f52-a204-441f0756bd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:25.878295 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b767b-eae9-4f52-a204-441f0756bd84 None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10/groups/a83da4807fb249adb95a024797878b3b/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.880413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b767b-eae9-4f52-a204-441f0756bd84 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8a10b282fb412fbc79062901f84a10, group_id=a83da4807fb249adb95a024797878b3b, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:25.882039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b767b-eae9-4f52-a204-441f0756bd84 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:25.893131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:25.931419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1509e9b-cb54-450a-a7a0-12c1a5609f7b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:25.952871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 201/406] 10.222.0.64 () {58 vars in 1442 bytes} [Mon May 23 15:21:25 2022] PUT /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10/groups/a83da4807fb249adb95a024797878b3b/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:21:25.962625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a291de-e837-4f1b-8e1b-c9825009624c None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:25.962625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a291de-e837-4f1b-8e1b-c9825009624c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:25.962625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a291de-e837-4f1b-8e1b-c9825009624c None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.962625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 202/407] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:25.963145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a291de-e837-4f1b-8e1b-c9825009624c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:25.965495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4912bf2-a3ed-484f-8d52-7f164f285279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:25.966402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4912bf2-a3ed-484f-8d52-7f164f285279 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:25.966689 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4912bf2-a3ed-484f-8d52-7f164f285279 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:25.966971 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4912bf2-a3ed-484f-8d52-7f164f285279 None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b/groups/a83da4807fb249adb95a024797878b3b/roles/1bac0428b2544e2cb16fad539f080c14` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:25.969549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4912bf2-a3ed-484f-8d52-7f164f285279 None admin] RBAC: Authorizing `identity:create_grant(project_id=936684d8d665414698e461a87200ef9b, group_id=a83da4807fb249adb95a024797878b3b, role_id=1bac0428b2544e2cb16fad539f080c14)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:25.971276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4912bf2-a3ed-484f-8d52-7f164f285279 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:25.991093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:26.057290 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 206/408] 10.222.0.64 () {58 vars in 1442 bytes} [Mon May 23 15:21:25 2022] PUT /identity/v3/projects/936684d8d665414698e461a87200ef9b/groups/a83da4807fb249adb95a024797878b3b/roles/1bac0428b2544e2cb16fad539f080c14 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:21:26.058994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e2f88-3e2e-4da5-9ebe-39fe37f58abc None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:26.059351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e2f88-3e2e-4da5-9ebe-39fe37f58abc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:26.059747 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e2f88-3e2e-4da5-9ebe-39fe37f58abc None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:26.065110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 207/409] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:26.065110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9639e9-7ada-4482-824a-1e3729458127 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:26.065110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9639e9-7ada-4482-824a-1e3729458127 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:26.065110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9639e9-7ada-4482-824a-1e3729458127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:26.085388 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1e9639e9-7ada-4482-824a-1e3729458127 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:26.086777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1e9639e9-7ada-4482-824a-1e3729458127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:26.090543 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a291de-e837-4f1b-8e1b-c9825009624c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:26.137325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7cd5c7-5700-4ec3-8dfb-a1dc8f0677d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:26.138366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cd5c7-5700-4ec3-8dfb-a1dc8f0677d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:26.138655 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cd5c7-5700-4ec3-8dfb-a1dc8f0677d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:26.138937 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cd5c7-5700-4ec3-8dfb-a1dc8f0677d9 None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10/groups/a83da4807fb249adb95a024797878b3b/roles/1bac0428b2544e2cb16fad539f080c14` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:26.141436 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7cd5c7-5700-4ec3-8dfb-a1dc8f0677d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8a10b282fb412fbc79062901f84a10, group_id=a83da4807fb249adb95a024797878b3b, role_id=1bac0428b2544e2cb16fad539f080c14)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:26.143115 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7cd5c7-5700-4ec3-8dfb-a1dc8f0677d9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:26.175452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 208/410] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:26.181407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbcb33-3981-4a90-9538-39aede8c7813 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:26.182082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbcb33-3981-4a90-9538-39aede8c7813 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:26.182579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbcb33-3981-4a90-9538-39aede8c7813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:26.203595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-47fbcb33-3981-4a90-9538-39aede8c7813 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:26.204878 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-47fbcb33-3981-4a90-9538-39aede8c7813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:26.216887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 203/411] 10.222.0.64 () {58 vars in 1442 bytes} [Mon May 23 15:21:25 2022] PUT /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10/groups/a83da4807fb249adb95a024797878b3b/roles/1bac0428b2544e2cb16fad539f080c14 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:21:26.217866 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7cd5c7-5700-4ec3-8dfb-a1dc8f0677d9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:26.288350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:26.325835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 209/412] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:26.354942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbcb33-3981-4a90-9538-39aede8c7813 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:26.360647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:26.390630 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7cd5c7-5700-4ec3-8dfb-a1dc8f0677d9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:26.448722 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd263e-4bdf-403c-ac98-1aedad176edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:26.449632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd263e-4bdf-403c-ac98-1aedad176edc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:26.449936 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd263e-4bdf-403c-ac98-1aedad176edc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:26.450193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd263e-4bdf-403c-ac98-1aedad176edc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:26.461781 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd263e-4bdf-403c-ac98-1aedad176edc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-0bdd263e-4bdf-403c-ac98-1aedad176edc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:26.469877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:26.470790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:26.471611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:26.471611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:26.471611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:26.471611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:26.471611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:26.471611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:26.471611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:26.471611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:26.471611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:26.471611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:26.471611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:26.471611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:26.471611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 204/413] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:26.477633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd263e-4bdf-403c-ac98-1aedad176edc None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:26.501488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbcb33-3981-4a90-9538-39aede8c7813 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:26.549639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75157f8-cd2f-4861-82d0-b5a4e58384df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:26.550923 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75157f8-cd2f-4861-82d0-b5a4e58384df None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:26.551869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75157f8-cd2f-4861-82d0-b5a4e58384df None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:26.552659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75157f8-cd2f-4861-82d0-b5a4e58384df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:26.566365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75157f8-cd2f-4861-82d0-b5a4e58384df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d75157f8-cd2f-4861-82d0-b5a4e58384df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:26.577627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:26.578672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:26.579474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:26.579474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:26.579474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:26.579474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:26.579474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:26.579474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:26.579474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:26.579474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:26.579474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:26.579474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:26.579474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:21:26.579474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:26.582266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 210/414] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:21:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:21:26.588734 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75157f8-cd2f-4861-82d0-b5a4e58384df None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:26.640801 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd263e-4bdf-403c-ac98-1aedad176edc None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:26.645668 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:26.731073 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75157f8-cd2f-4861-82d0-b5a4e58384df None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:26.733381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d75c21-6594-4f69-9226-aa7c9e9fa200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:26.734494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d75c21-6594-4f69-9226-aa7c9e9fa200 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:26.735542 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d75c21-6594-4f69-9226-aa7c9e9fa200 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:26.735964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d75c21-6594-4f69-9226-aa7c9e9fa200 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:26.737077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d75c21-6594-4f69-9226-aa7c9e9fa200 None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:26.740291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d75c21-6594-4f69-9226-aa7c9e9fa200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:26.741904 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d75c21-6594-4f69-9226-aa7c9e9fa200 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:26.759186 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 205/415] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:21:26.764703 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d75c21-6594-4f69-9226-aa7c9e9fa200 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:26.767893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:26.794719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b272d1-9e62-4330-836e-8d714849b980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:26.795546 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b272d1-9e62-4330-836e-8d714849b980 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:26.796024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b272d1-9e62-4330-836e-8d714849b980 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:26.796367 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b272d1-9e62-4330-836e-8d714849b980 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:26.797197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b272d1-9e62-4330-836e-8d714849b980 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:26.797605 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b272d1-9e62-4330-836e-8d714849b980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:26.799092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b272d1-9e62-4330-836e-8d714849b980 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:26.809650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 211/416] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:26.822131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b272d1-9e62-4330-836e-8d714849b980 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:26.919936 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d75c21-6594-4f69-9226-aa7c9e9fa200 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:26.983746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b272d1-9e62-4330-836e-8d714849b980 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:26.988698 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130008d-96bd-4248-99e4-009fe7250f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:26.990711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130008d-96bd-4248-99e4-009fe7250f6e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:26.991150 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:26.991918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130008d-96bd-4248-99e4-009fe7250f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:26.992341 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130008d-96bd-4248-99e4-009fe7250f6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:27.053558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f7020-4f8d-4ed5-a0fb-49666d129253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:27.054714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f7020-4f8d-4ed5-a0fb-49666d129253 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:27.056001 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f7020-4f8d-4ed5-a0fb-49666d129253 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:27.056422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f7020-4f8d-4ed5-a0fb-49666d129253 None admin] PATH_INFO: `/v3/groups/7869dc6bbe714697bfd76b940253cf6a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:27.058994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f7020-4f8d-4ed5-a0fb-49666d129253 None admin] RBAC: Authorizing `identity:get_group(group_id=7869dc6bbe714697bfd76b940253cf6a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:27.062435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f7020-4f8d-4ed5-a0fb-49666d129253 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:27.065049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 212/417] 10.222.0.64 () {56 vars in 1180 bytes} [Mon May 23 15:21:26 2022] GET /identity/v3/groups/7869dc6bbe714697bfd76b940253cf6a => generated 229 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:27.071690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f7020-4f8d-4ed5-a0fb-49666d129253 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:27.138818 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130008d-96bd-4248-99e4-009fe7250f6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:27.140302 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130008d-96bd-4248-99e4-009fe7250f6e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:27.148633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:27.280736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f7020-4f8d-4ed5-a0fb-49666d129253 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:27.286739 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 206/418] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:27.291922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130008d-96bd-4248-99e4-009fe7250f6e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:27.354531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414017f1-6a5d-47a0-804a-f60cdf407b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:27.355397 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414017f1-6a5d-47a0-804a-f60cdf407b96 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:27.355686 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414017f1-6a5d-47a0-804a-f60cdf407b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:27.355968 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414017f1-6a5d-47a0-804a-f60cdf407b96 None admin] PATH_INFO: `/v3/projects/4c9cfa8acbce47f6b0259cbfbf5b338d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:27.361870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414017f1-6a5d-47a0-804a-f60cdf407b96 None admin] RBAC: Authorizing `identity:get_project(project_id=4c9cfa8acbce47f6b0259cbfbf5b338d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:27.363597 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414017f1-6a5d-47a0-804a-f60cdf407b96 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:27.365714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 213/419] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:27 2022] GET /identity/v3/projects/4c9cfa8acbce47f6b0259cbfbf5b338d => generated 344 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:21:27.371366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414017f1-6a5d-47a0-804a-f60cdf407b96 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:27.405585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130008d-96bd-4248-99e4-009fe7250f6e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:27.407100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:27.443084 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f39d79-fe32-45bf-ab18-b49edf71acc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:27.443924 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f39d79-fe32-45bf-ab18-b49edf71acc4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:27.444211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f39d79-fe32-45bf-ab18-b49edf71acc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:27.444488 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f39d79-fe32-45bf-ab18-b49edf71acc4 None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:27.446425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f39d79-fe32-45bf-ab18-b49edf71acc4 None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:27.447886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f39d79-fe32-45bf-ab18-b49edf71acc4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:27.454047 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 207/420] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:27 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:21:27.454966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f39d79-fe32-45bf-ab18-b49edf71acc4 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:27.457819 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:27.568825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414017f1-6a5d-47a0-804a-f60cdf407b96 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:27.623705 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26780f0b-c4da-477e-9636-23a182867b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:27.624604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26780f0b-c4da-477e-9636-23a182867b66 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:27.624899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26780f0b-c4da-477e-9636-23a182867b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:27.625208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26780f0b-c4da-477e-9636-23a182867b66 None admin] PATH_INFO: `/v3/projects/4c9cfa8acbce47f6b0259cbfbf5b338d/groups/7869dc6bbe714697bfd76b940253cf6a/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:27.631742 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26780f0b-c4da-477e-9636-23a182867b66 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c9cfa8acbce47f6b0259cbfbf5b338d, group_id=7869dc6bbe714697bfd76b940253cf6a, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:27.634555 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26780f0b-c4da-477e-9636-23a182867b66 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:27.652914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f39d79-fe32-45bf-ab18-b49edf71acc4 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:27.689656 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7925165-74ce-42ac-b604-cb9759a7cbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:27.690556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7925165-74ce-42ac-b604-cb9759a7cbbe None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:27.690847 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7925165-74ce-42ac-b604-cb9759a7cbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:27.691132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7925165-74ce-42ac-b604-cb9759a7cbbe None admin] PATH_INFO: `/v3/groups/a83da4807fb249adb95a024797878b3b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:27.692501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7925165-74ce-42ac-b604-cb9759a7cbbe None admin] RBAC: Authorizing `identity:get_group(group_id=a83da4807fb249adb95a024797878b3b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:27.694294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7925165-74ce-42ac-b604-cb9759a7cbbe None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:27.700032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 208/421] 10.222.0.64 () {56 vars in 1180 bytes} [Mon May 23 15:21:27 2022] GET /identity/v3/groups/a83da4807fb249adb95a024797878b3b => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:27.702661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7925165-74ce-42ac-b604-cb9759a7cbbe None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:27.748211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:27.760018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 214/422] 10.222.0.64 () {58 vars in 1442 bytes} [Mon May 23 15:21:27 2022] PUT /identity/v3/projects/4c9cfa8acbce47f6b0259cbfbf5b338d/groups/7869dc6bbe714697bfd76b940253cf6a/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:21:27.825039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7925165-74ce-42ac-b604-cb9759a7cbbe None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:27.863288 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588a40e-3873-40e4-911d-d2a1f5b91751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:27.863963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588a40e-3873-40e4-911d-d2a1f5b91751 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:27.864854 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588a40e-3873-40e4-911d-d2a1f5b91751 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:27.865354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588a40e-3873-40e4-911d-d2a1f5b91751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:27.866214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588a40e-3873-40e4-911d-d2a1f5b91751 None admin] RBAC: Adding query filter params (group.id=a83da4807fb249adb95a024797878b3b, role.id=9330473f62ec4a82a82d1c4beb6160d7, scope.project.id=936684d8d665414698e461a87200ef9b) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:27.866535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588a40e-3873-40e4-911d-d2a1f5b91751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:27.868012 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588a40e-3873-40e4-911d-d2a1f5b91751 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:27.880986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 209/423] 10.222.0.64 () {56 vars in 1376 bytes} [Mon May 23 15:21:27 2022] GET /identity/v3/role_assignments?group.id=a83da4807fb249adb95a024797878b3b&scope.project.id=936684d8d665414698e461a87200ef9b&role.id=9330473f62ec4a82a82d1c4beb6160d7 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:21:28.510274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d827-b465-4a8a-8ead-0f21230ed5c4 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:28.510646 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d827-b465-4a8a-8ead-0f21230ed5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:28.510860 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d827-b465-4a8a-8ead-0f21230ed5c4 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:28.512360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 215/424] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:28.518025 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a1d61-a562-419a-99c3-4be5c42eb77c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:28.518861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a1d61-a562-419a-99c3-4be5c42eb77c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:28.519337 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a1d61-a562-419a-99c3-4be5c42eb77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:28.538147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-533a1d61-a562-419a-99c3-4be5c42eb77c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:28.539688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-533a1d61-a562-419a-99c3-4be5c42eb77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:28.613590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 210/425] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:28.638134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400d827-b465-4a8a-8ead-0f21230ed5c4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:28.720522 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a6056-c4ba-4fa2-9e0c-9055a746c120 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:28.721526 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a6056-c4ba-4fa2-9e0c-9055a746c120 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:28.722003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a6056-c4ba-4fa2-9e0c-9055a746c120 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:28.724347 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 211/426] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:21:28.730224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070c96d-b963-460e-af0c-aeb151ff1d73 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:28.730792 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070c96d-b963-460e-af0c-aeb151ff1d73 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:28.731298 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070c96d-b963-460e-af0c-aeb151ff1d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:28.753428 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c070c96d-b963-460e-af0c-aeb151ff1d73 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:28.755945 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c070c96d-b963-460e-af0c-aeb151ff1d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:28.778428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:28.782740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400d827-b465-4a8a-8ead-0f21230ed5c4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:28.795935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:28.831499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65648509-a767-4b88-bcde-816f48419e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:28.832544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65648509-a767-4b88-bcde-816f48419e89 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:28.833119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65648509-a767-4b88-bcde-816f48419e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:28.833538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65648509-a767-4b88-bcde-816f48419e89 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:28.845723 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65648509-a767-4b88-bcde-816f48419e89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:28.852442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 212/427] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-65648509-a767-4b88-bcde-816f48419e89 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:28.855224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:28.856130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:28.856947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:28.856947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:28.856947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:28.856947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:28.856947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:28.856947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:28.856947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:28.856947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:28.856947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:28.856947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:28.856947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:21:28.856947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:28.859556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 216/428] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:28.865529 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070c96d-b963-460e-af0c-aeb151ff1d73 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:28.877692 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65648509-a767-4b88-bcde-816f48419e89 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:28.980345 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070c96d-b963-460e-af0c-aeb151ff1d73 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:28.995180 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65648509-a767-4b88-bcde-816f48419e89 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:29.018130 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2021c5-887f-4475-acd4-3166bb8221f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:29.019092 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2021c5-887f-4475-acd4-3166bb8221f4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:29.019512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2021c5-887f-4475-acd4-3166bb8221f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:29.019798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2021c5-887f-4475-acd4-3166bb8221f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:29.021327 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2021c5-887f-4475-acd4-3166bb8221f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:29.021644 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2021c5-887f-4475-acd4-3166bb8221f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:29.023261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2021c5-887f-4475-acd4-3166bb8221f4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:29.030282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d9e31-e2c4-4932-a83b-37722ff3a769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:29.031094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d9e31-e2c4-4932-a83b-37722ff3a769 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:29.031382 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d9e31-e2c4-4932-a83b-37722ff3a769 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:29.031663 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d9e31-e2c4-4932-a83b-37722ff3a769 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:29.036320 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 213/429] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:21:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:29.037847 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a411f-5616-4751-a4c0-7195d495042f None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:29.038140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a411f-5616-4751-a4c0-7195d495042f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:29.038421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a411f-5616-4751-a4c0-7195d495042f None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:29.043965 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 214/430] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:29.043965 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a411f-5616-4751-a4c0-7195d495042f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:29.044139 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d9e31-e2c4-4932-a83b-37722ff3a769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-439d9e31-e2c4-4932-a83b-37722ff3a769 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:29.050710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:29.052152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:29.053237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:29.053237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:29.053237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:29.053237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:29.053237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:29.053237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:29.053237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:29.053237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:29.053237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:29.053237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:29.053237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:29.053237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:29.057342 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 217/431] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:21:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:21:29.058565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78129b-28c3-45fb-9bf8-1bc792dc209c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:29.058857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78129b-28c3-45fb-9bf8-1bc792dc209c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:29.059133 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78129b-28c3-45fb-9bf8-1bc792dc209c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:29.073107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:29.077206 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9e78129b-28c3-45fb-9bf8-1bc792dc209c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:29.078409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9e78129b-28c3-45fb-9bf8-1bc792dc209c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:29.092627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:29.152739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 218/432] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:29.153991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78129b-28c3-45fb-9bf8-1bc792dc209c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:29.157929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a411f-5616-4751-a4c0-7195d495042f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:29.200482 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26486d9a-ab93-4fab-8b63-9c45432f26ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:29.201174 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26486d9a-ab93-4fab-8b63-9c45432f26ea None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:29.201467 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26486d9a-ab93-4fab-8b63-9c45432f26ea None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:29.201745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26486d9a-ab93-4fab-8b63-9c45432f26ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:29.264506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78129b-28c3-45fb-9bf8-1bc792dc209c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:29.269583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26486d9a-ab93-4fab-8b63-9c45432f26ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:29.271105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26486d9a-ab93-4fab-8b63-9c45432f26ea None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:29.295745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:29.299566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dfa09-9b81-43c0-beaf-41dd07ea9fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:29.300235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dfa09-9b81-43c0-beaf-41dd07ea9fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:29.300523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dfa09-9b81-43c0-beaf-41dd07ea9fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:29.300802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dfa09-9b81-43c0-beaf-41dd07ea9fd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:29.302205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20dfa09-9b81-43c0-beaf-41dd07ea9fd7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:29.302630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20dfa09-9b81-43c0-beaf-41dd07ea9fd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:29.304265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20dfa09-9b81-43c0-beaf-41dd07ea9fd7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:29.312204 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 219/433] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:29.313004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dfa09-9b81-43c0-beaf-41dd07ea9fd7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:29.331318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:29.382597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 215/434] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:29.383430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26486d9a-ab93-4fab-8b63-9c45432f26ea None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:29.420545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dfa09-9b81-43c0-beaf-41dd07ea9fd7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:29.454108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160dcc5c-3b95-4d14-8ed1-766bacd798f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:29.454921 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160dcc5c-3b95-4d14-8ed1-766bacd798f7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:29.455213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160dcc5c-3b95-4d14-8ed1-766bacd798f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:29.455525 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160dcc5c-3b95-4d14-8ed1-766bacd798f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:29.467131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160dcc5c-3b95-4d14-8ed1-766bacd798f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-160dcc5c-3b95-4d14-8ed1-766bacd798f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:29.476506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:29.477431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:29.478275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:29.478275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:29.478275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:29.478275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:29.478275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:29.478275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:29.478275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:29.478275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:29.478275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:29.478275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:29.478275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:29.478275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:29.478275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 220/435] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:21:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:29.478949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160dcc5c-3b95-4d14-8ed1-766bacd798f7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:29.498215 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26486d9a-ab93-4fab-8b63-9c45432f26ea None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:29.524295 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:29.535221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c973a-bfbc-4108-80c4-e067fe728f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:29.536545 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c973a-bfbc-4108-80c4-e067fe728f75 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:29.536940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c973a-bfbc-4108-80c4-e067fe728f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:29.537280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c973a-bfbc-4108-80c4-e067fe728f75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:29.551423 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:29.582157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160dcc5c-3b95-4d14-8ed1-766bacd798f7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:29.609258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c973a-bfbc-4108-80c4-e067fe728f75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:29.610816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c973a-bfbc-4108-80c4-e067fe728f75 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:29.619541 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7cffcd-bdaf-4e89-bb23-52b54df2f3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:29.620463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cffcd-bdaf-4e89-bb23-52b54df2f3dc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:29.620924 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cffcd-bdaf-4e89-bb23-52b54df2f3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:29.621261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cffcd-bdaf-4e89-bb23-52b54df2f3dc None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:29.622474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7cffcd-bdaf-4e89-bb23-52b54df2f3dc None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:29.623914 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7cffcd-bdaf-4e89-bb23-52b54df2f3dc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:29.625637 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 221/436] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:29 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:29.626566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7cffcd-bdaf-4e89-bb23-52b54df2f3dc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:29.735473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7cffcd-bdaf-4e89-bb23-52b54df2f3dc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:29.741050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 216/437] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:29.741979 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c973a-bfbc-4108-80c4-e067fe728f75 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:29.768405 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:29.772549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d36ec7-4c83-40f0-95a1-af303342c6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:29.773258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d36ec7-4c83-40f0-95a1-af303342c6ad None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:29.773553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d36ec7-4c83-40f0-95a1-af303342c6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:29.773832 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d36ec7-4c83-40f0-95a1-af303342c6ad None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:29.774568 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d36ec7-4c83-40f0-95a1-af303342c6ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:29.774891 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d36ec7-4c83-40f0-95a1-af303342c6ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:29.776279 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d36ec7-4c83-40f0-95a1-af303342c6ad None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:29.785256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 222/438] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:29.787837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d36ec7-4c83-40f0-95a1-af303342c6ad None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:29.791418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:29.856846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c973a-bfbc-4108-80c4-e067fe728f75 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:29.890804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c237b-a465-4d5d-96de-bfa7d29b095d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:29.891667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c237b-a465-4d5d-96de-bfa7d29b095d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:29.891945 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c237b-a465-4d5d-96de-bfa7d29b095d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:29.892223 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c237b-a465-4d5d-96de-bfa7d29b095d None admin] PATH_INFO: `/v3/groups/a83da4807fb249adb95a024797878b3b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:29.893420 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c237b-a465-4d5d-96de-bfa7d29b095d None admin] RBAC: Authorizing `identity:get_group(group_id=a83da4807fb249adb95a024797878b3b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:29.894850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c237b-a465-4d5d-96de-bfa7d29b095d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:29.895017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d36ec7-4c83-40f0-95a1-af303342c6ad None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:29.901823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 217/439] 10.222.0.64 () {56 vars in 1180 bytes} [Mon May 23 15:21:29 2022] GET /identity/v3/groups/a83da4807fb249adb95a024797878b3b => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:29.901823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c237b-a465-4d5d-96de-bfa7d29b095d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:29.936033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af101a-d5b7-4f0f-83fb-4ca86517c71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:29.936900 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af101a-d5b7-4f0f-83fb-4ca86517c71a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:29.937223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af101a-d5b7-4f0f-83fb-4ca86517c71a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:29.937505 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af101a-d5b7-4f0f-83fb-4ca86517c71a None admin] PATH_INFO: `/v3/projects/936684d8d665414698e461a87200ef9b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:29.938753 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af101a-d5b7-4f0f-83fb-4ca86517c71a None admin] RBAC: Authorizing `identity:get_project(project_id=936684d8d665414698e461a87200ef9b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:29.940207 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af101a-d5b7-4f0f-83fb-4ca86517c71a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:29.941931 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 223/440] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:29 2022] GET /identity/v3/projects/936684d8d665414698e461a87200ef9b => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:29.942753 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af101a-d5b7-4f0f-83fb-4ca86517c71a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:30.000815 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:30.011047 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c237b-a465-4d5d-96de-bfa7d29b095d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:30.015234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:30.050073 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af101a-d5b7-4f0f-83fb-4ca86517c71a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:30.054281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3e569-5364-4434-ba21-54001018026c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:30.054968 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3e569-5364-4434-ba21-54001018026c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:30.055259 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3e569-5364-4434-ba21-54001018026c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:30.055544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3e569-5364-4434-ba21-54001018026c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:30.090521 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d9201-4d0f-422e-a8dd-b8e0c980e3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:30.091190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d9201-4d0f-422e-a8dd-b8e0c980e3fd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:30.091476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d9201-4d0f-422e-a8dd-b8e0c980e3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:30.091758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d9201-4d0f-422e-a8dd-b8e0c980e3fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:30.092618 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d9201-4d0f-422e-a8dd-b8e0c980e3fd None admin] RBAC: Adding query filter params (group.id=a83da4807fb249adb95a024797878b3b, role.id=9330473f62ec4a82a82d1c4beb6160d7, scope.project.id=bc8a10b282fb412fbc79062901f84a10) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:30.092907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d9201-4d0f-422e-a8dd-b8e0c980e3fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:30.094360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d9201-4d0f-422e-a8dd-b8e0c980e3fd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:30.109131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 224/441] 10.222.0.64 () {56 vars in 1376 bytes} [Mon May 23 15:21:29 2022] GET /identity/v3/role_assignments?group.id=a83da4807fb249adb95a024797878b3b&scope.project.id=bc8a10b282fb412fbc79062901f84a10&role.id=9330473f62ec4a82a82d1c4beb6160d7 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:30.109131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d9201-4d0f-422e-a8dd-b8e0c980e3fd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:30.141706 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3e569-5364-4434-ba21-54001018026c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:30.143317 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3e569-5364-4434-ba21-54001018026c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:30.243190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d9201-4d0f-422e-a8dd-b8e0c980e3fd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:30.247381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:30.257863 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 218/442] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:30.261537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8e978-2d12-4321-bac8-be7af61822b7 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:30.261537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8e978-2d12-4321-bac8-be7af61822b7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:30.261537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8e978-2d12-4321-bac8-be7af61822b7 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:30.262372 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 219/443] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:30.263812 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8e978-2d12-4321-bac8-be7af61822b7 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:30.283325 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b13f9-d600-45bf-a219-f6a3dd7dbc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:30.285092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b13f9-d600-45bf-a219-f6a3dd7dbc03 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:30.285608 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b13f9-d600-45bf-a219-f6a3dd7dbc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:30.286794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b13f9-d600-45bf-a219-f6a3dd7dbc03 None admin] PATH_INFO: `/v3/groups/a83da4807fb249adb95a024797878b3b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:30.288293 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b13f9-d600-45bf-a219-f6a3dd7dbc03 None admin] RBAC: Authorizing `identity:get_group(group_id=a83da4807fb249adb95a024797878b3b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:30.289938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b13f9-d600-45bf-a219-f6a3dd7dbc03 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:30.291782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 225/444] 10.222.0.64 () {56 vars in 1180 bytes} [Mon May 23 15:21:30 2022] GET /identity/v3/groups/a83da4807fb249adb95a024797878b3b => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:30.293178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd9ec6-67d9-4755-a688-e21b450889f7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:30.293627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd9ec6-67d9-4755-a688-e21b450889f7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:30.293913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd9ec6-67d9-4755-a688-e21b450889f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:30.299819 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:30.313448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1abd9ec6-67d9-4755-a688-e21b450889f7 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:30.314640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1abd9ec6-67d9-4755-a688-e21b450889f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:30.387972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8e978-2d12-4321-bac8-be7af61822b7 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:30.390019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 226/445] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:30.390858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd9ec6-67d9-4755-a688-e21b450889f7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:30.429446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a476b-5c72-466f-9c23-a351067cbcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:30.430308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a476b-5c72-466f-9c23-a351067cbcb9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:30.430591 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a476b-5c72-466f-9c23-a351067cbcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:30.430864 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a476b-5c72-466f-9c23-a351067cbcb9 None admin] PATH_INFO: `/v3/projects/bc8a10b282fb412fbc79062901f84a10` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:30.432031 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a476b-5c72-466f-9c23-a351067cbcb9 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8a10b282fb412fbc79062901f84a10)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:30.433505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a476b-5c72-466f-9c23-a351067cbcb9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:30.435184 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 220/446] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:30 2022] GET /identity/v3/projects/bc8a10b282fb412fbc79062901f84a10 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:30.435937 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a476b-5c72-466f-9c23-a351067cbcb9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:30.490724 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd9ec6-67d9-4755-a688-e21b450889f7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:30.495540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:30.519554 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:30.525246 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24032a-263f-47e7-84e1-28db0f1dcdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:30.525907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24032a-263f-47e7-84e1-28db0f1dcdd7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:30.526198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24032a-263f-47e7-84e1-28db0f1dcdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:30.526477 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24032a-263f-47e7-84e1-28db0f1dcdd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:30.527309 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24032a-263f-47e7-84e1-28db0f1dcdd7 None admin] RBAC: Adding query filter params (group.id=a83da4807fb249adb95a024797878b3b, role.id=1bac0428b2544e2cb16fad539f080c14, scope.project.id=936684d8d665414698e461a87200ef9b) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:30.527628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24032a-263f-47e7-84e1-28db0f1dcdd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:30.529143 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24032a-263f-47e7-84e1-28db0f1dcdd7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:30.534807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 227/447] 10.222.0.64 () {56 vars in 1376 bytes} [Mon May 23 15:21:30 2022] GET /identity/v3/role_assignments?group.id=a83da4807fb249adb95a024797878b3b&scope.project.id=936684d8d665414698e461a87200ef9b&role.id=1bac0428b2544e2cb16fad539f080c14 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:30.535583 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24032a-263f-47e7-84e1-28db0f1dcdd7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:30.539352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a476b-5c72-466f-9c23-a351067cbcb9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:30.584516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490da1a1-327b-478b-98a6-05d7f5a59bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:30.585358 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490da1a1-327b-478b-98a6-05d7f5a59bca None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:30.585639 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490da1a1-327b-478b-98a6-05d7f5a59bca None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:30.585912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490da1a1-327b-478b-98a6-05d7f5a59bca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:30.598971 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490da1a1-327b-478b-98a6-05d7f5a59bca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-490da1a1-327b-478b-98a6-05d7f5a59bca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:30.607929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:30.609189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:30.610754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:30.610754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:30.610754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:30.610754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:30.610754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:30.610754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:30.610754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:30.610754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:30.610754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:30.610754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:30.610754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:21:30.610754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:30.610754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 221/448] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:21:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:30.616674 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490da1a1-327b-478b-98a6-05d7f5a59bca None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:30.663326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24032a-263f-47e7-84e1-28db0f1dcdd7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:30.711862 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443267d-a6a8-4578-9939-7b36e626c745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:30.712740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443267d-a6a8-4578-9939-7b36e626c745 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:30.713028 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443267d-a6a8-4578-9939-7b36e626c745 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:30.713337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443267d-a6a8-4578-9939-7b36e626c745 None admin] PATH_INFO: `/v3/groups/a83da4807fb249adb95a024797878b3b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:30.714530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443267d-a6a8-4578-9939-7b36e626c745 None admin] RBAC: Authorizing `identity:get_group(group_id=a83da4807fb249adb95a024797878b3b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:30.716305 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443267d-a6a8-4578-9939-7b36e626c745 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:30.718139 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 228/449] 10.222.0.64 () {56 vars in 1180 bytes} [Mon May 23 15:21:30 2022] GET /identity/v3/groups/a83da4807fb249adb95a024797878b3b => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:30.724349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443267d-a6a8-4578-9939-7b36e626c745 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:30.741435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490da1a1-327b-478b-98a6-05d7f5a59bca None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:30.765312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:30.785126 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c582a0-45fe-486b-825f-c9a24ce73e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:30.785876 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c582a0-45fe-486b-825f-c9a24ce73e0d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:30.786171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c582a0-45fe-486b-825f-c9a24ce73e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:30.786454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c582a0-45fe-486b-825f-c9a24ce73e0d None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:30.787214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c582a0-45fe-486b-825f-c9a24ce73e0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:30.791029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c582a0-45fe-486b-825f-c9a24ce73e0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:30.791029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c582a0-45fe-486b-825f-c9a24ce73e0d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:30.797955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 222/450] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:21:30.799510 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:30.804687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c582a0-45fe-486b-825f-c9a24ce73e0d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:30.845979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443267d-a6a8-4578-9939-7b36e626c745 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:30.888708 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d02cd-5987-4737-8762-bc295cedfd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:30.889494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d02cd-5987-4737-8762-bc295cedfd8d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:30.889782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d02cd-5987-4737-8762-bc295cedfd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:30.890056 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d02cd-5987-4737-8762-bc295cedfd8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:30.890881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d02cd-5987-4737-8762-bc295cedfd8d None admin] RBAC: Adding query filter params (group.id=a83da4807fb249adb95a024797878b3b, role.id=1bac0428b2544e2cb16fad539f080c14, scope.project.id=bc8a10b282fb412fbc79062901f84a10) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:30.891204 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d02cd-5987-4737-8762-bc295cedfd8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:30.892632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d02cd-5987-4737-8762-bc295cedfd8d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:30.899474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 229/451] 10.222.0.64 () {56 vars in 1376 bytes} [Mon May 23 15:21:30 2022] GET /identity/v3/role_assignments?group.id=a83da4807fb249adb95a024797878b3b&scope.project.id=bc8a10b282fb412fbc79062901f84a10&role.id=1bac0428b2544e2cb16fad539f080c14 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:21:30.929177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c582a0-45fe-486b-825f-c9a24ce73e0d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:30.966423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813ebab-6ccd-4053-bb08-542f77190d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:30.967112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813ebab-6ccd-4053-bb08-542f77190d09 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:30.967400 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813ebab-6ccd-4053-bb08-542f77190d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:30.967681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813ebab-6ccd-4053-bb08-542f77190d09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:31.028512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:31.050482 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813ebab-6ccd-4053-bb08-542f77190d09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:31.052587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813ebab-6ccd-4053-bb08-542f77190d09 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:31.163705 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 223/452] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:21:31.169472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d02cd-5987-4737-8762-bc295cedfd8d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:31.273812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d02cd-5987-4737-8762-bc295cedfd8d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:31.298995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:31.307257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f6816-dc42-4884-9fad-43b1e7d40992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:31.308112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f6816-dc42-4884-9fad-43b1e7d40992 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:31.308402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f6816-dc42-4884-9fad-43b1e7d40992 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:31.308684 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f6816-dc42-4884-9fad-43b1e7d40992 None admin] PATH_INFO: `/v3/projects/4c9cfa8acbce47f6b0259cbfbf5b338d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:31.309898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f6816-dc42-4884-9fad-43b1e7d40992 None admin] RBAC: Authorizing `identity:get_project(project_id=4c9cfa8acbce47f6b0259cbfbf5b338d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:31.311347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f6816-dc42-4884-9fad-43b1e7d40992 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:31.313030 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 230/453] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:21:31 2022] GET /identity/v3/projects/4c9cfa8acbce47f6b0259cbfbf5b338d => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:31.319402 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813ebab-6ccd-4053-bb08-542f77190d09 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:31.396917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:31.414301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813ebab-6ccd-4053-bb08-542f77190d09 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:31.447455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07bcaeb-9cbf-4a84-9577-3f66e0847c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:31.448305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07bcaeb-9cbf-4a84-9577-3f66e0847c75 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:31.448594 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07bcaeb-9cbf-4a84-9577-3f66e0847c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:31.448874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07bcaeb-9cbf-4a84-9577-3f66e0847c75 None admin] PATH_INFO: `/v3/groups/7869dc6bbe714697bfd76b940253cf6a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:31.450057 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07bcaeb-9cbf-4a84-9577-3f66e0847c75 None admin] RBAC: Authorizing `identity:get_group(group_id=7869dc6bbe714697bfd76b940253cf6a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:31.451519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07bcaeb-9cbf-4a84-9577-3f66e0847c75 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:31.453140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 224/454] 10.222.0.64 () {56 vars in 1180 bytes} [Mon May 23 15:21:31 2022] GET /identity/v3/groups/7869dc6bbe714697bfd76b940253cf6a => generated 229 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:31.459398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f6816-dc42-4884-9fad-43b1e7d40992 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:31.555040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f6816-dc42-4884-9fad-43b1e7d40992 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:31.586635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb0d8f-1962-4eb5-aa6e-deea867f9af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:31.587265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb0d8f-1962-4eb5-aa6e-deea867f9af3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:31.587548 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb0d8f-1962-4eb5-aa6e-deea867f9af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:31.587859 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb0d8f-1962-4eb5-aa6e-deea867f9af3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:31.588668 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb0d8f-1962-4eb5-aa6e-deea867f9af3 None admin] RBAC: Adding query filter params (group.id=7869dc6bbe714697bfd76b940253cf6a, role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=4c9cfa8acbce47f6b0259cbfbf5b338d) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:31.588977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb0d8f-1962-4eb5-aa6e-deea867f9af3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:31.590410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb0d8f-1962-4eb5-aa6e-deea867f9af3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:31.596052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 231/455] 10.222.0.64 () {56 vars in 1376 bytes} [Mon May 23 15:21:31 2022] GET /identity/v3/role_assignments?group.id=7869dc6bbe714697bfd76b940253cf6a&scope.project.id=4c9cfa8acbce47f6b0259cbfbf5b338d&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:34.346619 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c3a5d-1a8f-4223-9c7b-70f2b21442bc None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:34.347552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c3a5d-1a8f-4223-9c7b-70f2b21442bc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:34.347982 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c3a5d-1a8f-4223-9c7b-70f2b21442bc None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:34.351159 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 225/456] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:34.362921 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56fc71-dfff-47bf-97ee-0384893d611c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:34.364663 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56fc71-dfff-47bf-97ee-0384893d611c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:34.365345 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56fc71-dfff-47bf-97ee-0384893d611c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:34.390906 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7f56fc71-dfff-47bf-97ee-0384893d611c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:34.392366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7f56fc71-dfff-47bf-97ee-0384893d611c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:34.471716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:34.483089 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 232/457] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:21:34.526519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c3a5d-1a8f-4223-9c7b-70f2b21442bc None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:34.584091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1e36e-48e2-4315-ae8a-7d2baa171476 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:34.584414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1e36e-48e2-4315-ae8a-7d2baa171476 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:34.584645 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1e36e-48e2-4315-ae8a-7d2baa171476 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:34.586592 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 233/458] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:21:34.592316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583c4b9-c324-4b06-9d50-27bfea53b500 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:34.592924 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583c4b9-c324-4b06-9d50-27bfea53b500 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:34.593633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583c4b9-c324-4b06-9d50-27bfea53b500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:34.630175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2583c4b9-c324-4b06-9d50-27bfea53b500 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:34.631285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2583c4b9-c324-4b06-9d50-27bfea53b500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:34.687293 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c3a5d-1a8f-4223-9c7b-70f2b21442bc None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:34.732858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3668ecb-9051-4c0a-86bf-b1d60e5805c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:34.733938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3668ecb-9051-4c0a-86bf-b1d60e5805c1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:34.734374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3668ecb-9051-4c0a-86bf-b1d60e5805c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:34.734799 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3668ecb-9051-4c0a-86bf-b1d60e5805c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:34.741333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 234/459] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:34.742915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fb10a-b678-4811-898f-553073418337 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:34.743211 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fb10a-b678-4811-898f-553073418337 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:34.743613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fb10a-b678-4811-898f-553073418337 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:34.745393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 235/460] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:34.746698 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3668ecb-9051-4c0a-86bf-b1d60e5805c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:34.748322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3668ecb-9051-4c0a-86bf-b1d60e5805c1 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:34.755625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aef3b5-4046-4d0b-a27b-e5d00a81e8e1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:34.755912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aef3b5-4046-4d0b-a27b-e5d00a81e8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:34.756194 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aef3b5-4046-4d0b-a27b-e5d00a81e8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:34.768433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-a3668ecb-9051-4c0a-86bf-b1d60e5805c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:34.768433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:34.768433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:34.768433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:34.768433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:34.768433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:34.768433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:34.768433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:34.768433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:34.768433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:34.768433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:34.768433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:34.768433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:34.768433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:34.768433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:34.769540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:34.770756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:34.770756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:34.770756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:34.770756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:34.772532 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 226/461] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:34.773515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3668ecb-9051-4c0a-86bf-b1d60e5805c1 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:34.777730 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:34.788090 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-25aef3b5-4046-4d0b-a27b-e5d00a81e8e1 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:34.789615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-25aef3b5-4046-4d0b-a27b-e5d00a81e8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:34.877314 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 236/462] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 23 15:21:34.878175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aef3b5-4046-4d0b-a27b-e5d00a81e8e1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:34.886039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3668ecb-9051-4c0a-86bf-b1d60e5805c1 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:34.924243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2752c-d59d-4683-b31c-42cdde1a867b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:34.925039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2752c-d59d-4683-b31c-42cdde1a867b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:34.925355 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2752c-d59d-4683-b31c-42cdde1a867b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:34.925637 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2752c-d59d-4683-b31c-42cdde1a867b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:34.928457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:34.937192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2752c-d59d-4683-b31c-42cdde1a867b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:34.938912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2752c-d59d-4683-b31c-42cdde1a867b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:34.958328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-ccf2752c-d59d-4683-b31c-42cdde1a867b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:34.958328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:34.958328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:34.958328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:34.958328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:34.958328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:34.958328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:34.958328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:34.958328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:34.958328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:34.958328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:34.958328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:34.958328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:34.958328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:34.958328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:34.959228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:34.960192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:34.960192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:34.960192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:34.960192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:34.960192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 227/463] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:34.961232 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8ff08-0bf7-4281-8562-1bd70165509e None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:34.961232 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8ff08-0bf7-4281-8562-1bd70165509e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:34.961389 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8ff08-0bf7-4281-8562-1bd70165509e None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:34.965152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 228/464] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:34.965152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8ff08-0bf7-4281-8562-1bd70165509e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:34.987850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aef3b5-4046-4d0b-a27b-e5d00a81e8e1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:35.027114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:35.027334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a59450-9c10-4112-9e9b-0db93dba7ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:35.028196 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a59450-9c10-4112-9e9b-0db93dba7ddf None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:35.028773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a59450-9c10-4112-9e9b-0db93dba7ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:35.028773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a59450-9c10-4112-9e9b-0db93dba7ddf None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:35.029528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a59450-9c10-4112-9e9b-0db93dba7ddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:35.029843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a59450-9c10-4112-9e9b-0db93dba7ddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:35.031214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a59450-9c10-4112-9e9b-0db93dba7ddf None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:35.047632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 237/465] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:35.047632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a59450-9c10-4112-9e9b-0db93dba7ddf None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:35.076617 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8ff08-0bf7-4281-8562-1bd70165509e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:35.110366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d3da2-2213-47eb-ad31-65f0412d4bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:35.111297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d3da2-2213-47eb-ad31-65f0412d4bec None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:35.111586 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d3da2-2213-47eb-ad31-65f0412d4bec None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:35.111870 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d3da2-2213-47eb-ad31-65f0412d4bec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:35.123118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d3da2-2213-47eb-ad31-65f0412d4bec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:35.124567 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d3da2-2213-47eb-ad31-65f0412d4bec None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:35.143913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-808d3da2-2213-47eb-ad31-65f0412d4bec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:35.143913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:35.143913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:35.143913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:35.143913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:35.143913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:35.143913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:35.143913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:35.143913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:35.143913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:35.143913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:35.143913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:35.143913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:35.143913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:35.143913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:35.144802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:35.145728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:35.145728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:35.145728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:35.145728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:35.145728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 229/466] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:35.146860 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c454fd-3c11-49ae-aa38-6dac82fd910e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:35.147145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c454fd-3c11-49ae-aa38-6dac82fd910e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:35.147422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c454fd-3c11-49ae-aa38-6dac82fd910e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:35.158482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a59450-9c10-4112-9e9b-0db93dba7ddf None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:35.164095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-44c454fd-3c11-49ae-aa38-6dac82fd910e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:35.165348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-44c454fd-3c11-49ae-aa38-6dac82fd910e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:35.169215 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:35.193794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc8e46-60e5-41b0-bd6b-f690fec20504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:35.194467 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc8e46-60e5-41b0-bd6b-f690fec20504 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:35.194760 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc8e46-60e5-41b0-bd6b-f690fec20504 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:35.195046 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc8e46-60e5-41b0-bd6b-f690fec20504 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:35.195785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc8e46-60e5-41b0-bd6b-f690fec20504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:35.196099 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc8e46-60e5-41b0-bd6b-f690fec20504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:35.197525 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc8e46-60e5-41b0-bd6b-f690fec20504 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:35.211020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 238/467] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:35.211830 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc8e46-60e5-41b0-bd6b-f690fec20504 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:35.241981 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 230/468] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:35.242664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c454fd-3c11-49ae-aa38-6dac82fd910e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:35.286786 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:35.335887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc8e46-60e5-41b0-bd6b-f690fec20504 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:35.360475 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c454fd-3c11-49ae-aa38-6dac82fd910e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:35.374052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785576e-11aa-4517-923d-053e9561b8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:35.375015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785576e-11aa-4517-923d-053e9561b8bb None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:35.375310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785576e-11aa-4517-923d-053e9561b8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:35.375587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785576e-11aa-4517-923d-053e9561b8bb None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:35.376463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785576e-11aa-4517-923d-053e9561b8bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:35.378315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785576e-11aa-4517-923d-053e9561b8bb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:35.399371 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72a37a-371e-4ec5-b3e9-309860493355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:35.399962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72a37a-371e-4ec5-b3e9-309860493355 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:35.400244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72a37a-371e-4ec5-b3e9-309860493355 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:35.400522 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72a37a-371e-4ec5-b3e9-309860493355 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:35.401291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72a37a-371e-4ec5-b3e9-309860493355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:35.401630 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72a37a-371e-4ec5-b3e9-309860493355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:35.403087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72a37a-371e-4ec5-b3e9-309860493355 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:35.421045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 231/469] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:35.421848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72a37a-371e-4ec5-b3e9-309860493355 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:35.480535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 239/470] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:21:35 2022] POST /identity/v3/users => generated 246 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:21:35.481489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785576e-11aa-4517-923d-053e9561b8bb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:35.519920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:35.536714 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:35.543820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72a37a-371e-4ec5-b3e9-309860493355 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:35.592156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6cbaf-3d91-4e4d-8edd-ec3fb8639da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:35.592862 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6cbaf-3d91-4e4d-8edd-ec3fb8639da0 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:35.593380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6cbaf-3d91-4e4d-8edd-ec3fb8639da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:35.593763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6cbaf-3d91-4e4d-8edd-ec3fb8639da0 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:35.594545 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6cbaf-3d91-4e4d-8edd-ec3fb8639da0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:35.596066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6cbaf-3d91-4e4d-8edd-ec3fb8639da0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:35.622696 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785576e-11aa-4517-923d-053e9561b8bb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:35.667337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c09252-43b4-4cb8-97cf-77c1a16714fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:35.668065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c09252-43b4-4cb8-97cf-77c1a16714fc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:35.668354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c09252-43b4-4cb8-97cf-77c1a16714fc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:35.668633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c09252-43b4-4cb8-97cf-77c1a16714fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:35.680943 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c09252-43b4-4cb8-97cf-77c1a16714fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:35.682445 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c09252-43b4-4cb8-97cf-77c1a16714fc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:35.702462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-50c09252-43b4-4cb8-97cf-77c1a16714fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:35.702462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:35.702462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:35.702462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:35.702462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:35.702462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:35.702462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:35.702462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:35.702462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:35.702462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:35.702462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:35.702462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:35.702462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:35.702462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:35.702462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:35.703432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:35.704291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:35.704291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:35.704291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:35.704291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:35.704291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 240/471] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:35.704975 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c09252-43b4-4cb8-97cf-77c1a16714fc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:35.792126 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 232/472] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:21:35 2022] POST /identity/v3/users => generated 248 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:21:35.793523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6cbaf-3d91-4e4d-8edd-ec3fb8639da0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:35.798028 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:35.822093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c09252-43b4-4cb8-97cf-77c1a16714fc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:35.863439 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1e28b-0259-49b0-bff1-6b32d2e75c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:35.864676 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1e28b-0259-49b0-bff1-6b32d2e75c23 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:35.864976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1e28b-0259-49b0-bff1-6b32d2e75c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:35.865335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1e28b-0259-49b0-bff1-6b32d2e75c23 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:35.866352 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1e28b-0259-49b0-bff1-6b32d2e75c23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:35.867989 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1e28b-0259-49b0-bff1-6b32d2e75c23 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:35.920151 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6cbaf-3d91-4e4d-8edd-ec3fb8639da0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:35.957034 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcdf90-bc13-4fa6-8bfd-e8da614fd5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:35.958470 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcdf90-bc13-4fa6-8bfd-e8da614fd5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:35.958768 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcdf90-bc13-4fa6-8bfd-e8da614fd5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:35.959050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcdf90-bc13-4fa6-8bfd-e8da614fd5c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:35.959794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcdf90-bc13-4fa6-8bfd-e8da614fd5c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:35.960109 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcdf90-bc13-4fa6-8bfd-e8da614fd5c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:35.961547 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcdf90-bc13-4fa6-8bfd-e8da614fd5c8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:35.982063 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 233/474] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:35.982063 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a47a80-f4cb-4a46-8185-8df823e71415 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:35.982063 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a47a80-f4cb-4a46-8185-8df823e71415 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:35.982063 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a47a80-f4cb-4a46-8185-8df823e71415 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:35.982063 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 234/475] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:35.982799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 241/475] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:21:35 2022] POST /identity/v3/users => generated 248 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:21:35.987092 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a47a80-f4cb-4a46-8185-8df823e71415 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:35.990547 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bebfec-ef37-4096-ba49-72fd81380e1b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:35.991185 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bebfec-ef37-4096-ba49-72fd81380e1b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:35.991887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bebfec-ef37-4096-ba49-72fd81380e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:36.009746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:36.026803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-60bebfec-ef37-4096-ba49-72fd81380e1b None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:36.028473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-60bebfec-ef37-4096-ba49-72fd81380e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:36.120015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a47a80-f4cb-4a46-8185-8df823e71415 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:36.121781 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 242/476] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:36.155397 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bebfec-ef37-4096-ba49-72fd81380e1b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:36.169003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af137815-7686-4104-b775-89b6bc527cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:36.169789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af137815-7686-4104-b775-89b6bc527cf4 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:36.170084 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af137815-7686-4104-b775-89b6bc527cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:36.170433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af137815-7686-4104-b775-89b6bc527cf4 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:36.171109 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af137815-7686-4104-b775-89b6bc527cf4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:36.172665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af137815-7686-4104-b775-89b6bc527cf4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:36.233572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:36.277225 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 235/477] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:21:35 2022] POST /identity/v3/users => generated 249 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:21:36.281006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bebfec-ef37-4096-ba49-72fd81380e1b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:36.327575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4372b-a6b4-412c-8735-d5a0b2db19c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:36.328313 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4372b-a6b4-412c-8735-d5a0b2db19c0 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:36.328599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4372b-a6b4-412c-8735-d5a0b2db19c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:36.328874 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4372b-a6b4-412c-8735-d5a0b2db19c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:36.329670 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4372b-a6b4-412c-8735-d5a0b2db19c0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:36.331083 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4372b-a6b4-412c-8735-d5a0b2db19c0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:36.370100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.sql.core [None req-94a4372b-a6b4-412c-8735-d5a0b2db19c0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 15:21:36.370100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 15:21:36.370100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [parameters: {'id': 'acd9a254ebad4f6697804a81b0ce5f6d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 15:21:36.370100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80599) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 15:21:36.389100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-94a4372b-a6b4-412c-8735-d5a0b2db19c0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 15:21:36.389100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:36.389100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:21:36.389100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:21:36.389100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:21:36.389100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:21:36.389100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:21:36.389100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:21:36.389100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:21:36.389100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application conn.query(q) May 23 15:21:36.389100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:21:36.389100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:21:36.389100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:21:36.389100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application result.read() May 23 15:21:36.389100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return ref.to_dict() May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:21:36.390620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application next(self.gen) May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application yield resource May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application next(self.gen) May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self.session.rollback() May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self.force_reraise() May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise self.value May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application session.commit() May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 15:21:36.391533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._prepare_impl() May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self.session.flush() May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._flush(objects) May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application compat.raise_( May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application flush_context.execute() May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rec.execute(self) May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 15:21:36.393870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:21:36.397528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application conn.query(q) May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application result.read() May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:21:36.399454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [parameters: {'id': 'acd9a254ebad4f6697804a81b0ce5f6d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:36.401365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 15:21:36.402479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:36.403515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 243/478] 10.222.0.64 () {60 vars in 1131 bytes} [Mon May 23 15:21:36 2022] POST /identity/v3/roles => generated 144 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:36.403592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af137815-7686-4104-b775-89b6bc527cf4 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:36.498772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:36.537600 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af137815-7686-4104-b775-89b6bc527cf4 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:36.572768 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb55b99-0397-4a34-96d2-f6599bf1b2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:36.573618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb55b99-0397-4a34-96d2-f6599bf1b2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:36.573952 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb55b99-0397-4a34-96d2-f6599bf1b2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:36.574238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb55b99-0397-4a34-96d2-f6599bf1b2d5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:36.586803 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb55b99-0397-4a34-96d2-f6599bf1b2d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-0bb55b99-0397-4a34-96d2-f6599bf1b2d5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:36.595373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:36.601351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:36.602473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:36.602473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:36.602473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:36.602473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:36.602473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:36.602473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:36.602473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:36.602473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:36.602473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:36.602473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:36.602473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:21:36.602473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:36.602473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 236/479] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:21:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:36.605007 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4372b-a6b4-412c-8735-d5a0b2db19c0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:36.731610 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4372b-a6b4-412c-8735-d5a0b2db19c0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:36.759548 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:36.770711 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470a3e0-9b31-42df-aa32-10020027b266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:36.771397 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470a3e0-9b31-42df-aa32-10020027b266 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:36.771681 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470a3e0-9b31-42df-aa32-10020027b266 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:36.771960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470a3e0-9b31-42df-aa32-10020027b266 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:36.775818 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470a3e0-9b31-42df-aa32-10020027b266 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:36.775818 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470a3e0-9b31-42df-aa32-10020027b266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:36.775818 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470a3e0-9b31-42df-aa32-10020027b266 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:36.784594 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 244/480] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:21:38.052981 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2ae66-6a11-4277-a082-06a7836a16c3 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:38.053537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2ae66-6a11-4277-a082-06a7836a16c3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:38.053614 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2ae66-6a11-4277-a082-06a7836a16c3 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:38.055223 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 237/481] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:38.060680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5981dae-24d1-4e48-b363-f4f5f7f08451 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:38.061097 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5981dae-24d1-4e48-b363-f4f5f7f08451 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:38.061283 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5981dae-24d1-4e48-b363-f4f5f7f08451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:38.081320 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d5981dae-24d1-4e48-b363-f4f5f7f08451 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:38.083159 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d5981dae-24d1-4e48-b363-f4f5f7f08451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:38.179827 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 245/482] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:21:38.219734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2ae66-6a11-4277-a082-06a7836a16c3 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:38.372490 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2ae66-6a11-4277-a082-06a7836a16c3 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:38.400699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:38.409152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d9f62-a685-4ee6-b5a4-5d83eef5e1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:38.409961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d9f62-a685-4ee6-b5a4-5d83eef5e1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:38.410793 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d9f62-a685-4ee6-b5a4-5d83eef5e1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:38.415888 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d9f62-a685-4ee6-b5a4-5d83eef5e1c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:38.425581 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d9f62-a685-4ee6-b5a4-5d83eef5e1c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-b15d9f62-a685-4ee6-b5a4-5d83eef5e1c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:38.433955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:38.435050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:38.436073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:38.436073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:38.436073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:38.436073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:38.436073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:38.436073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:38.436073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:38.436073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:38.436073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:38.436073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:38.436073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:38.436073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:38.443329 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 238/483] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:38.449507 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5981dae-24d1-4e48-b363-f4f5f7f08451 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:38.498821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b662f-c6ab-4ea1-9d87-d61d3858dfdc None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:38.499108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b662f-c6ab-4ea1-9d87-d61d3858dfdc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:38.499393 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b662f-c6ab-4ea1-9d87-d61d3858dfdc None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:38.501155 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 239/484] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:21:38.508657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1016c5-3e27-450e-8017-f4fd960fad38 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:38.508950 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1016c5-3e27-450e-8017-f4fd960fad38 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:38.509260 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1016c5-3e27-450e-8017-f4fd960fad38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:38.528192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2e1016c5-3e27-450e-8017-f4fd960fad38 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:38.529416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2e1016c5-3e27-450e-8017-f4fd960fad38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:38.602547 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5981dae-24d1-4e48-b363-f4f5f7f08451 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:38.638859 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:38.641917 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd236977-8ba8-49bd-824c-ecf6f7e62d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:38.642591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd236977-8ba8-49bd-824c-ecf6f7e62d57 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:38.642916 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd236977-8ba8-49bd-824c-ecf6f7e62d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:38.643189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd236977-8ba8-49bd-824c-ecf6f7e62d57 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:38.643931 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd236977-8ba8-49bd-824c-ecf6f7e62d57 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:38.644237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd236977-8ba8-49bd-824c-ecf6f7e62d57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:38.645899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd236977-8ba8-49bd-824c-ecf6f7e62d57 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:38.655855 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 246/485] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:21:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:38.661903 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd236977-8ba8-49bd-824c-ecf6f7e62d57 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:38.672668 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 240/486] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:21:38.702008 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1016c5-3e27-450e-8017-f4fd960fad38 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:38.793558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd236977-8ba8-49bd-824c-ecf6f7e62d57 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:38.838553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1016c5-3e27-450e-8017-f4fd960fad38 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:38.860274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9e130-9942-4a0c-8b6f-bd9cb288747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:38.861452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9e130-9942-4a0c-8b6f-bd9cb288747d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:38.861939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9e130-9942-4a0c-8b6f-bd9cb288747d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:38.862411 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9e130-9942-4a0c-8b6f-bd9cb288747d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:38.869178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:38.883576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9e130-9942-4a0c-8b6f-bd9cb288747d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:38.886094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9e130-9942-4a0c-8b6f-bd9cb288747d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:38.904376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538145c0-64a1-4137-ab8d-8194ed7851d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:38.905524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538145c0-64a1-4137-ab8d-8194ed7851d0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:38.906015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538145c0-64a1-4137-ab8d-8194ed7851d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:38.906516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538145c0-64a1-4137-ab8d-8194ed7851d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:38.916752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-fce9e130-9942-4a0c-8b6f-bd9cb288747d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:38.916752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:38.916752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:38.916752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:38.916752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:38.916752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:38.916752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:38.916752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:38.916752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:38.916752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:38.916752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:38.916752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:38.916752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:38.916752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:38.916752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:38.917634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:38.918551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:38.918551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:38.918551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:38.918551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:38.925591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 247/487] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:38.930361 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538145c0-64a1-4137-ab8d-8194ed7851d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:38.938023 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9e130-9942-4a0c-8b6f-bd9cb288747d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-538145c0-64a1-4137-ab8d-8194ed7851d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:38.948744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:38.949731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:38.950544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:38.950544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:38.950544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:38.950544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:38.950544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:38.950544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:38.950544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:38.950544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:38.950544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:38.950544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:38.950544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:38.950544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:38.955036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 241/488] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:21:38.961660 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538145c0-64a1-4137-ab8d-8194ed7851d0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:39.017470 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:39.064779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9e130-9942-4a0c-8b6f-bd9cb288747d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:39.095620 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538145c0-64a1-4137-ab8d-8194ed7851d0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:39.101513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8752453-dde9-43c9-b980-b8c1819b2b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:39.102318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8752453-dde9-43c9-b980-b8c1819b2b53 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:39.102712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8752453-dde9-43c9-b980-b8c1819b2b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:39.103096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8752453-dde9-43c9-b980-b8c1819b2b53 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:39.103930 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8752453-dde9-43c9-b980-b8c1819b2b53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:39.104346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8752453-dde9-43c9-b980-b8c1819b2b53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:39.105875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8752453-dde9-43c9-b980-b8c1819b2b53 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:39.119688 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 248/489] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:39.121282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d3fa8-3b8e-4db0-8681-46c3f0b33133 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:39.121648 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d3fa8-3b8e-4db0-8681-46c3f0b33133 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:39.122002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d3fa8-3b8e-4db0-8681-46c3f0b33133 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:39.127694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 249/490] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:39.128667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d3fa8-3b8e-4db0-8681-46c3f0b33133 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:39.140342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b546f2-cde6-4cfa-bc95-29193a0229ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:39.141336 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b546f2-cde6-4cfa-bc95-29193a0229ff None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:39.142206 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b546f2-cde6-4cfa-bc95-29193a0229ff None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:39.142944 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b546f2-cde6-4cfa-bc95-29193a0229ff None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:39.143942 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b546f2-cde6-4cfa-bc95-29193a0229ff None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:39.144771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b546f2-cde6-4cfa-bc95-29193a0229ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:39.147343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b546f2-cde6-4cfa-bc95-29193a0229ff None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:39.159616 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 242/491] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:21:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:21:39.162194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cc928-7900-419f-aca2-e26b21a68387 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:39.163063 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cc928-7900-419f-aca2-e26b21a68387 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:39.163495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cc928-7900-419f-aca2-e26b21a68387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:39.185568 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b96cc928-7900-419f-aca2-e26b21a68387 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:39.187039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b96cc928-7900-419f-aca2-e26b21a68387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:39.241217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:39.249426 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d3fa8-3b8e-4db0-8681-46c3f0b33133 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:39.253462 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:39.288889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec04ad2-1fc1-453a-9c7a-d6de9223239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:39.290173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec04ad2-1fc1-453a-9c7a-d6de9223239e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:39.290728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec04ad2-1fc1-453a-9c7a-d6de9223239e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:39.291200 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec04ad2-1fc1-453a-9c7a-d6de9223239e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:39.293306 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 243/492] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:39.295304 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cc928-7900-419f-aca2-e26b21a68387 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:39.301395 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec04ad2-1fc1-453a-9c7a-d6de9223239e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:39.303517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec04ad2-1fc1-453a-9c7a-d6de9223239e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6ec04ad2-1fc1-453a-9c7a-d6de9223239e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:39.331813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:39.333486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:39.334408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:39.334408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:39.334408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:39.334408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:39.334408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:39.334408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:39.334408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:39.334408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:39.334408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:21:39.334408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:39.337153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 250/493] 10.222.0.64 () {56 vars in 1093 bytes} [Mon May 23 15:21:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:39.338278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec04ad2-1fc1-453a-9c7a-d6de9223239e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:39.441998 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cc928-7900-419f-aca2-e26b21a68387 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:39.444895 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec04ad2-1fc1-453a-9c7a-d6de9223239e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:39.478039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96083f0f-eadf-42f4-90fc-2dcb837dda7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:39.478849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96083f0f-eadf-42f4-90fc-2dcb837dda7c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:39.479052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2794a0-e301-4250-ab17-bc39a9f857a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:39.479165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96083f0f-eadf-42f4-90fc-2dcb837dda7c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:39.479400 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96083f0f-eadf-42f4-90fc-2dcb837dda7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:39.479851 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2794a0-e301-4250-ab17-bc39a9f857a2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:39.480137 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2794a0-e301-4250-ab17-bc39a9f857a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:39.480412 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2794a0-e301-4250-ab17-bc39a9f857a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:39.490850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96083f0f-eadf-42f4-90fc-2dcb837dda7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:39.491040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2794a0-e301-4250-ab17-bc39a9f857a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:39.492329 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96083f0f-eadf-42f4-90fc-2dcb837dda7c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-7d2794a0-e301-4250-ab17-bc39a9f857a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:39.498694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:39.499576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:39.500370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:39.500370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:39.500370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:39.500370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:39.500370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:39.500370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:39.500370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:39.500370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:39.500370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:39.500370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:39.500370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:39.500370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:39.500370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 251/494] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:39.501031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2794a0-e301-4250-ab17-bc39a9f857a2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:39.516587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-96083f0f-eadf-42f4-90fc-2dcb837dda7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:39.516587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:39.516587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:39.516587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:39.516587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:39.516587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:39.516587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:39.516587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:39.516587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:39.516587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:39.516587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:39.516587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:39.516587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:39.516587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:39.516587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:39.517696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:39.518587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:39.518587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:39.518587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:39.518587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:39.518587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 244/495] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:39.519305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e557e-ef60-47da-8bf8-39a1305f2724 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:39.519489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e557e-ef60-47da-8bf8-39a1305f2724 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:39.520037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e557e-ef60-47da-8bf8-39a1305f2724 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:39.525937 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 245/496] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:39.525937 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e557e-ef60-47da-8bf8-39a1305f2724 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:39.526217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:39.559069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:39.620133 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2794a0-e301-4250-ab17-bc39a9f857a2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:39.657499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70925f-83a5-4acd-b4e1-a92d80bdb704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:39.658176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70925f-83a5-4acd-b4e1-a92d80bdb704 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:39.658460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70925f-83a5-4acd-b4e1-a92d80bdb704 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:39.658740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70925f-83a5-4acd-b4e1-a92d80bdb704 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:39.659582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70925f-83a5-4acd-b4e1-a92d80bdb704 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:39.659893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70925f-83a5-4acd-b4e1-a92d80bdb704 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:39.661590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70925f-83a5-4acd-b4e1-a92d80bdb704 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:39.689197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e557e-ef60-47da-8bf8-39a1305f2724 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:39.706356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 252/497] 10.222.0.64 () {56 vars in 1133 bytes} [Mon May 23 15:21:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:39.707904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70925f-83a5-4acd-b4e1-a92d80bdb704 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:39.741894 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e89069-0b80-484e-88eb-56019a79d9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:39.743099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e89069-0b80-484e-88eb-56019a79d9dc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:39.743454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e89069-0b80-484e-88eb-56019a79d9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:39.743902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e89069-0b80-484e-88eb-56019a79d9dc None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:39.745237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e89069-0b80-484e-88eb-56019a79d9dc None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:39.745686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e89069-0b80-484e-88eb-56019a79d9dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:39.748309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e89069-0b80-484e-88eb-56019a79d9dc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:39.764825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 246/498] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:21:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:39.764825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a702d-b8bf-48c5-b4d5-3b8429b7c13b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:39.764825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a702d-b8bf-48c5-b4d5-3b8429b7c13b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:39.764825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a702d-b8bf-48c5-b4d5-3b8429b7c13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:39.789723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e16a702d-b8bf-48c5-b4d5-3b8429b7c13b None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:39.791746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e16a702d-b8bf-48c5-b4d5-3b8429b7c13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:39.806101 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:39.838345 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70925f-83a5-4acd-b4e1-a92d80bdb704 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:39.875136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6882fd-3bb3-44ba-8926-fd088e15bb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:39.875812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6882fd-3bb3-44ba-8926-fd088e15bb25 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:39.876099 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6882fd-3bb3-44ba-8926-fd088e15bb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:39.876389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6882fd-3bb3-44ba-8926-fd088e15bb25 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:39.877152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6882fd-3bb3-44ba-8926-fd088e15bb25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:39.877463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6882fd-3bb3-44ba-8926-fd088e15bb25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:39.878850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6882fd-3bb3-44ba-8926-fd088e15bb25 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:39.885917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 247/499] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:39.886893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a702d-b8bf-48c5-b4d5-3b8429b7c13b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:39.902123 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:39.908527 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 253/500] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:39.908527 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6882fd-3bb3-44ba-8926-fd088e15bb25 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:39.988822 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a702d-b8bf-48c5-b4d5-3b8429b7c13b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:40.007051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6882fd-3bb3-44ba-8926-fd088e15bb25 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:40.022792 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab84fdf-345f-49be-9f92-a680ff590fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:40.023447 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab84fdf-345f-49be-9f92-a680ff590fbb None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:40.023731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab84fdf-345f-49be-9f92-a680ff590fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:40.024006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab84fdf-345f-49be-9f92-a680ff590fbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:40.036232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:40.041081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb86b3-1416-42de-9e74-6df535d39fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:40.041916 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb86b3-1416-42de-9e74-6df535d39fbd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:40.042261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb86b3-1416-42de-9e74-6df535d39fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:40.042550 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb86b3-1416-42de-9e74-6df535d39fbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:40.053455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb86b3-1416-42de-9e74-6df535d39fbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:40.055063 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb86b3-1416-42de-9e74-6df535d39fbd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:40.073925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-cefb86b3-1416-42de-9e74-6df535d39fbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:40.073925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:40.073925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:40.073925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:40.073925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:40.073925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:40.073925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:40.073925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:40.073925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:40.073925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:40.073925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:40.073925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:40.073925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:40.073925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:40.073925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:40.075301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:40.076175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:40.076175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:40.076175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:40.076175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:40.076175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 254/501] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:40.076460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13541e-f252-4dbc-93e5-c267e0b37865 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:40.076762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13541e-f252-4dbc-93e5-c267e0b37865 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:40.077006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13541e-f252-4dbc-93e5-c267e0b37865 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:40.078304 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 255/502] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:40.079001 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13541e-f252-4dbc-93e5-c267e0b37865 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:40.096288 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab84fdf-345f-49be-9f92-a680ff590fbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:40.097922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab84fdf-345f-49be-9f92-a680ff590fbb None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:40.112240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:40.181753 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13541e-f252-4dbc-93e5-c267e0b37865 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:40.200815 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 248/503] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:40.201690 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab84fdf-345f-49be-9f92-a680ff590fbb None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:40.220558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622680e9-0ed2-4218-a0bd-c3e2655f5bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:40.221614 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622680e9-0ed2-4218-a0bd-c3e2655f5bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:40.221899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622680e9-0ed2-4218-a0bd-c3e2655f5bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:40.222174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622680e9-0ed2-4218-a0bd-c3e2655f5bc2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:40.229899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622680e9-0ed2-4218-a0bd-c3e2655f5bc2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:40.231355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622680e9-0ed2-4218-a0bd-c3e2655f5bc2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-622680e9-0ed2-4218-a0bd-c3e2655f5bc2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:40.245452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:40.246407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:40.247323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:40.247323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:40.247323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:40.247323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:40.247323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:40.247323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:40.247323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:40.247323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:40.247323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 15:21:40.247323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:40.247323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 256/504] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:40.247323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622680e9-0ed2-4218-a0bd-c3e2655f5bc2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:40.292283 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:40.305796 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab84fdf-345f-49be-9f92-a680ff590fbb None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:40.320360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:40.339733 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d1fe0-b092-4d85-94af-00ef61f6e939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:40.340513 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d1fe0-b092-4d85-94af-00ef61f6e939 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:40.340804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d1fe0-b092-4d85-94af-00ef61f6e939 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:40.341107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d1fe0-b092-4d85-94af-00ef61f6e939 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:40.346910 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622680e9-0ed2-4218-a0bd-c3e2655f5bc2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:40.351771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d1fe0-b092-4d85-94af-00ef61f6e939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-a28d1fe0-b092-4d85-94af-00ef61f6e939 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:40.359446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:40.360465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:40.361351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:40.361351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:40.361351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:40.361351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:40.361351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:40.361351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:40.361351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:40.361351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:40.361351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:40.361351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:40.361351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:40.361351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:40.361351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 249/505] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:40.362034 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97691c5-571a-44eb-bf42-4bf06a475f77 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:40.362273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97691c5-571a-44eb-bf42-4bf06a475f77 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:40.362461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97691c5-571a-44eb-bf42-4bf06a475f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:40.379168 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b97691c5-571a-44eb-bf42-4bf06a475f77 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:40.380373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b97691c5-571a-44eb-bf42-4bf06a475f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:40.385863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecf9c7-8949-420b-9ed2-d4e99883b5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:40.386533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecf9c7-8949-420b-9ed2-d4e99883b5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:40.386822 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecf9c7-8949-420b-9ed2-d4e99883b5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:40.387105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecf9c7-8949-420b-9ed2-d4e99883b5f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:40.387833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecf9c7-8949-420b-9ed2-d4e99883b5f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:40.388149 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecf9c7-8949-420b-9ed2-d4e99883b5f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:40.389570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecf9c7-8949-420b-9ed2-d4e99883b5f2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:40.401895 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 257/506] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:40.402727 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecf9c7-8949-420b-9ed2-d4e99883b5f2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:40.447370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 250/507] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:40.448240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97691c5-571a-44eb-bf42-4bf06a475f77 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:40.505956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecf9c7-8949-420b-9ed2-d4e99883b5f2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:40.508189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:40.541161 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038eecff-2bdb-42cc-ba05-e124c28767ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:40.541965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eecff-2bdb-42cc-ba05-e124c28767ca None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:40.542256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eecff-2bdb-42cc-ba05-e124c28767ca None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:40.542541 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eecff-2bdb-42cc-ba05-e124c28767ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:40.551660 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97691c5-571a-44eb-bf42-4bf06a475f77 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:40.553254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038eecff-2bdb-42cc-ba05-e124c28767ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:40.554720 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038eecff-2bdb-42cc-ba05-e124c28767ca None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:40.555110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:40.572433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-038eecff-2bdb-42cc-ba05-e124c28767ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:40.572433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:40.572433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:40.572433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:40.572433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:40.572433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:40.572433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:40.572433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:40.572433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:40.572433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:40.572433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:40.572433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:40.572433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:40.572433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:40.572433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:40.573396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:40.574588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:40.574588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:40.574588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:40.574588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:40.574588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 258/508] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:40.574588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038eecff-2bdb-42cc-ba05-e124c28767ca None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:40.588603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53815bcf-bcb8-45ae-a476-b3083f6b0240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:40.589381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53815bcf-bcb8-45ae-a476-b3083f6b0240 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:40.589668 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53815bcf-bcb8-45ae-a476-b3083f6b0240 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:40.589951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53815bcf-bcb8-45ae-a476-b3083f6b0240 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:40.590790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53815bcf-bcb8-45ae-a476-b3083f6b0240 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:40.591100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53815bcf-bcb8-45ae-a476-b3083f6b0240 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:40.592551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53815bcf-bcb8-45ae-a476-b3083f6b0240 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:40.630536 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 251/509] 10.222.0.64 () {56 vars in 1137 bytes} [Mon May 23 15:21:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:40.631387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53815bcf-bcb8-45ae-a476-b3083f6b0240 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:40.683304 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038eecff-2bdb-42cc-ba05-e124c28767ca None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:40.719717 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0bf60-ac70-49af-98bf-31e7aef50103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:40.720388 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0bf60-ac70-49af-98bf-31e7aef50103 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:40.720691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0bf60-ac70-49af-98bf-31e7aef50103 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:40.720959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0bf60-ac70-49af-98bf-31e7aef50103 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:40.721737 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0bf60-ac70-49af-98bf-31e7aef50103 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:40.722050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0bf60-ac70-49af-98bf-31e7aef50103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:40.724311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0bf60-ac70-49af-98bf-31e7aef50103 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:40.732304 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 259/510] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:21:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:40.733148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0bf60-ac70-49af-98bf-31e7aef50103 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:40.733558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53815bcf-bcb8-45ae-a476-b3083f6b0240 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:40.752354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:40.771908 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5746a2-97ec-48e5-ad5c-b9986f39c27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:40.772781 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5746a2-97ec-48e5-ad5c-b9986f39c27d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:40.772984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5746a2-97ec-48e5-ad5c-b9986f39c27d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:40.773296 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5746a2-97ec-48e5-ad5c-b9986f39c27d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:40.780654 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5746a2-97ec-48e5-ad5c-b9986f39c27d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:40.782165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5746a2-97ec-48e5-ad5c-b9986f39c27d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-ed5746a2-97ec-48e5-ad5c-b9986f39c27d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:40.798113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:40.799207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:40.800140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:40.800140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:40.800140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:40.800140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:40.800140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:40.800140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:40.800140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:40.800140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:40.800140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:21:40.800140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:40.800140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 252/511] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:40.800140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5746a2-97ec-48e5-ad5c-b9986f39c27d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:40.826131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:40.838912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0bf60-ac70-49af-98bf-31e7aef50103 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:40.871768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff1695-92d8-4c3d-8f32-87ef0dd3fd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:40.872812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff1695-92d8-4c3d-8f32-87ef0dd3fd91 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:40.873088 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff1695-92d8-4c3d-8f32-87ef0dd3fd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:40.873372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff1695-92d8-4c3d-8f32-87ef0dd3fd91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:40.884491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff1695-92d8-4c3d-8f32-87ef0dd3fd91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:40.885967 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff1695-92d8-4c3d-8f32-87ef0dd3fd91 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:40.901412 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5746a2-97ec-48e5-ad5c-b9986f39c27d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:40.902577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f5ff1695-92d8-4c3d-8f32-87ef0dd3fd91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:40.902577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:40.902577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:40.902577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:40.902577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:40.902577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:40.902577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:40.902577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:40.902577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:40.902577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:40.902577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:40.902577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:40.902577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:40.902577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:40.902577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:40.903422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:40.904300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:40.904300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:40.904300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:40.904300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:40.904300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 260/512] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:40.904576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff1695-92d8-4c3d-8f32-87ef0dd3fd91 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:40.938277 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d7328-2589-4880-901a-43fd1bffc48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:40.938945 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d7328-2589-4880-901a-43fd1bffc48f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:40.939238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d7328-2589-4880-901a-43fd1bffc48f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:40.939521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d7328-2589-4880-901a-43fd1bffc48f None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:40.940244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d7328-2589-4880-901a-43fd1bffc48f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:40.940557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d7328-2589-4880-901a-43fd1bffc48f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:40.941984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d7328-2589-4880-901a-43fd1bffc48f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:40.953937 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 253/513] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:40.954923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d7328-2589-4880-901a-43fd1bffc48f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:40.987241 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:41.022955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff1695-92d8-4c3d-8f32-87ef0dd3fd91 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:41.036496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:41.053799 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d7328-2589-4880-901a-43fd1bffc48f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:41.057538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6deb0-5aeb-4f60-86d4-3ec3f8c8209f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:41.058202 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6deb0-5aeb-4f60-86d4-3ec3f8c8209f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:41.058495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6deb0-5aeb-4f60-86d4-3ec3f8c8209f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:41.058774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6deb0-5aeb-4f60-86d4-3ec3f8c8209f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:41.089672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114381a-eb4b-44ca-b85b-56588fee7e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:41.090461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114381a-eb4b-44ca-b85b-56588fee7e41 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:41.090752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114381a-eb4b-44ca-b85b-56588fee7e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:41.091022 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114381a-eb4b-44ca-b85b-56588fee7e41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:41.101710 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114381a-eb4b-44ca-b85b-56588fee7e41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:41.103185 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114381a-eb4b-44ca-b85b-56588fee7e41 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:41.120662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-7114381a-eb4b-44ca-b85b-56588fee7e41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:41.120662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:41.120662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:41.120662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:41.120662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:41.120662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:41.120662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:41.120662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:41.120662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:41.120662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:41.120662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:41.120662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:41.120662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:41.120662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:41.120662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:41.121772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:41.122858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:41.122858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:41.122858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:41.122858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:41.122858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 254/514] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:41.122858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114381a-eb4b-44ca-b85b-56588fee7e41 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:41.130353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6deb0-5aeb-4f60-86d4-3ec3f8c8209f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:41.131825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6deb0-5aeb-4f60-86d4-3ec3f8c8209f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:41.199499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:41.223396 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114381a-eb4b-44ca-b85b-56588fee7e41 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:41.233244 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 261/515] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:41.234534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6deb0-5aeb-4f60-86d4-3ec3f8c8209f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:41.258870 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44ab9f-62d7-40c0-ad1e-1c0e6b750d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:41.259548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44ab9f-62d7-40c0-ad1e-1c0e6b750d64 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:41.259835 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44ab9f-62d7-40c0-ad1e-1c0e6b750d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:41.260114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44ab9f-62d7-40c0-ad1e-1c0e6b750d64 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:41.260955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44ab9f-62d7-40c0-ad1e-1c0e6b750d64 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:41.261303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44ab9f-62d7-40c0-ad1e-1c0e6b750d64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:41.262742 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44ab9f-62d7-40c0-ad1e-1c0e6b750d64 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:41.264368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:41.281112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 255/516] 10.222.0.64 () {56 vars in 1137 bytes} [Mon May 23 15:21:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:41.281940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44ab9f-62d7-40c0-ad1e-1c0e6b750d64 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:41.348471 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6deb0-5aeb-4f60-86d4-3ec3f8c8209f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:41.383244 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6c524-6796-4c21-8083-6221933c6512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:41.383857 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44ab9f-62d7-40c0-ad1e-1c0e6b750d64 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:41.383983 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6c524-6796-4c21-8083-6221933c6512 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:41.384173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6c524-6796-4c21-8083-6221933c6512 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:41.384446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6c524-6796-4c21-8083-6221933c6512 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:41.385196 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6c524-6796-4c21-8083-6221933c6512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:41.385506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6c524-6796-4c21-8083-6221933c6512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:41.386875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6c524-6796-4c21-8083-6221933c6512 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:41.398107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 262/517] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:41.398949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6c524-6796-4c21-8083-6221933c6512 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:41.418605 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce383af-973c-46ce-8cd4-e155c69e26c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:41.419409 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce383af-973c-46ce-8cd4-e155c69e26c9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:41.419701 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce383af-973c-46ce-8cd4-e155c69e26c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:41.419983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce383af-973c-46ce-8cd4-e155c69e26c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:41.428185 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce383af-973c-46ce-8cd4-e155c69e26c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:41.429706 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce383af-973c-46ce-8cd4-e155c69e26c9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:41.433146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:41.444195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-cce383af-973c-46ce-8cd4-e155c69e26c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:41.444195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:41.444195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:41.444195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:41.444195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:41.444195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:41.444195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:41.444195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:41.444195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:41.444195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:41.444195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:41.444195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:41.444195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:21:41.444195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:21:41.444195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:41.449333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:41.450257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:21:41.450257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:21:41.450257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:21:41.450257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:21:41.450257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:21:41.450257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:21:41.450257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:41.450257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:41.450257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 256/518] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:41.450257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce383af-973c-46ce-8cd4-e155c69e26c9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:41.501613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:41.507128 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6c524-6796-4c21-8083-6221933c6512 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:41.540200 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101322c-5166-44fc-a8e8-c41dfe47e36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:41.540783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101322c-5166-44fc-a8e8-c41dfe47e36c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:41.541087 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101322c-5166-44fc-a8e8-c41dfe47e36c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:41.541380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101322c-5166-44fc-a8e8-c41dfe47e36c None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:41.542090 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101322c-5166-44fc-a8e8-c41dfe47e36c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:41.542401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101322c-5166-44fc-a8e8-c41dfe47e36c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:41.543786 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101322c-5166-44fc-a8e8-c41dfe47e36c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:41.548338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce383af-973c-46ce-8cd4-e155c69e26c9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:41.560652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 263/519] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:41.560652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101322c-5166-44fc-a8e8-c41dfe47e36c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:41.587464 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd22c9-8d1c-4657-9b6c-93b60e18d6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:41.588265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd22c9-8d1c-4657-9b6c-93b60e18d6fd None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:41.588556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd22c9-8d1c-4657-9b6c-93b60e18d6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:41.588840 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd22c9-8d1c-4657-9b6c-93b60e18d6fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:41.600225 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd22c9-8d1c-4657-9b6c-93b60e18d6fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:41.604108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd22c9-8d1c-4657-9b6c-93b60e18d6fd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:41.619738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-60dd22c9-8d1c-4657-9b6c-93b60e18d6fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:41.619738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:41.619738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:41.619738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:41.619738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:41.619738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:41.619738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:41.619738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:41.619738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:41.619738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:41.619738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:41.619738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:41.619738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:41.619738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:41.619738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:41.620851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:41.621764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:41.621764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:41.621764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:41.621764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:41.621764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 257/520] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:41.621764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd22c9-8d1c-4657-9b6c-93b60e18d6fd None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:41.648279 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:41.657287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101322c-5166-44fc-a8e8-c41dfe47e36c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:41.692807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070756c3-eb3e-44fc-bfaf-5aff16810099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:41.693960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070756c3-eb3e-44fc-bfaf-5aff16810099 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:41.694252 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070756c3-eb3e-44fc-bfaf-5aff16810099 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:41.694535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070756c3-eb3e-44fc-bfaf-5aff16810099 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:41.723715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd22c9-8d1c-4657-9b6c-93b60e18d6fd None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:41.726111 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:41.759052 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7b442-134b-453c-a6fb-51978b8b25ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:41.759724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7b442-134b-453c-a6fb-51978b8b25ca None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:41.760014 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7b442-134b-453c-a6fb-51978b8b25ca None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:41.760296 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7b442-134b-453c-a6fb-51978b8b25ca None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:41.761012 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7b442-134b-453c-a6fb-51978b8b25ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:41.762559 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7b442-134b-453c-a6fb-51978b8b25ca None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:41.763997 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070756c3-eb3e-44fc-bfaf-5aff16810099 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:41.765473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070756c3-eb3e-44fc-bfaf-5aff16810099 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:41.843900 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 258/521] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:21:41 2022] POST /identity/v3/users => generated 247 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:21:41.844687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7b442-134b-453c-a6fb-51978b8b25ca None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:41.854213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:41.865890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 264/522] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:41.866789 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070756c3-eb3e-44fc-bfaf-5aff16810099 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:41.980377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070756c3-eb3e-44fc-bfaf-5aff16810099 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:41.980638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7b442-134b-453c-a6fb-51978b8b25ca None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:42.018739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf1ea6-ec9f-45c7-b487-a8ed76477016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:42.018955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d24755-436f-448d-8120-a9fe9ca35280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:42.019600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf1ea6-ec9f-45c7-b487-a8ed76477016 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:42.019711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d24755-436f-448d-8120-a9fe9ca35280 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:42.019881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf1ea6-ec9f-45c7-b487-a8ed76477016 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:42.020512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf1ea6-ec9f-45c7-b487-a8ed76477016 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:42.020620 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d24755-436f-448d-8120-a9fe9ca35280 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:42.021092 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d24755-436f-448d-8120-a9fe9ca35280 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:42.022603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d24755-436f-448d-8120-a9fe9ca35280 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:42.022603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d24755-436f-448d-8120-a9fe9ca35280 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:42.025223 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d24755-436f-448d-8120-a9fe9ca35280 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:42.029584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf1ea6-ec9f-45c7-b487-a8ed76477016 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:42.031052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf1ea6-ec9f-45c7-b487-a8ed76477016 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:42.045326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 259/523] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:21:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:42.046046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d24755-436f-448d-8120-a9fe9ca35280 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-63cf1ea6-ec9f-45c7-b487-a8ed76477016 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:42.046864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:42.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:42.054340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:42.054340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:42.054340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:42.054340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:42.054340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:42.054340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:42.054340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:42.054340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:42.054340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:21:42.054340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:42.054340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 265/524] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:42.054340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf1ea6-ec9f-45c7-b487-a8ed76477016 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:42.081237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:42.120294 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:42.178523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d24755-436f-448d-8120-a9fe9ca35280 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:42.195831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf1ea6-ec9f-45c7-b487-a8ed76477016 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:42.213468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30f4be-a510-4df3-b4b5-c6017d3a2358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:42.214144 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30f4be-a510-4df3-b4b5-c6017d3a2358 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:42.214455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30f4be-a510-4df3-b4b5-c6017d3a2358 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:42.214721 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30f4be-a510-4df3-b4b5-c6017d3a2358 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:42.215443 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30f4be-a510-4df3-b4b5-c6017d3a2358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:42.215759 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30f4be-a510-4df3-b4b5-c6017d3a2358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:42.217182 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30f4be-a510-4df3-b4b5-c6017d3a2358 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:42.225314 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa8da0-3940-47e0-880b-f2025e944c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:42.226208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa8da0-3940-47e0-880b-f2025e944c1b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:42.226639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa8da0-3940-47e0-880b-f2025e944c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:42.230003 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa8da0-3940-47e0-880b-f2025e944c1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:42.235058 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 260/525] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:42.235058 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30f4be-a510-4df3-b4b5-c6017d3a2358 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:42.245515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa8da0-3940-47e0-880b-f2025e944c1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:42.246993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa8da0-3940-47e0-880b-f2025e944c1b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:42.264999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-dfaa8da0-3940-47e0-880b-f2025e944c1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:42.264999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:42.264999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:42.264999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:42.264999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:42.264999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:42.264999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:42.264999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:42.264999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:42.264999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:42.264999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:42.264999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:42.264999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:42.264999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:42.264999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:42.266152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:42.267033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:42.267033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:42.267033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:42.267033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:42.267033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 266/526] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:42.267033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa8da0-3940-47e0-880b-f2025e944c1b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:42.323447 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:42.343801 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30f4be-a510-4df3-b4b5-c6017d3a2358 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:42.376731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa8da0-3940-47e0-880b-f2025e944c1b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:42.378237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:42.381393 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f413b75-0692-41f5-bb2d-c8812259b18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:42.382075 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f413b75-0692-41f5-bb2d-c8812259b18b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:42.382366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f413b75-0692-41f5-bb2d-c8812259b18b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:42.382648 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f413b75-0692-41f5-bb2d-c8812259b18b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:42.383495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f413b75-0692-41f5-bb2d-c8812259b18b None admin] RBAC: Adding query filter params (role.id=3d6e986dd2264e3cb7e01e25dbac6f97, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=8da96bed73794fee8f719ec904936164) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:42.383871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f413b75-0692-41f5-bb2d-c8812259b18b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:42.385366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f413b75-0692-41f5-bb2d-c8812259b18b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:42.397121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 261/527] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:21:42 2022] GET /identity/v3/role_assignments?user.id=8da96bed73794fee8f719ec904936164&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=3d6e986dd2264e3cb7e01e25dbac6f97 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:42.397121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f413b75-0692-41f5-bb2d-c8812259b18b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:42.434873 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da5312-4315-4519-a5c4-7ee5120b5d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:42.435765 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da5312-4315-4519-a5c4-7ee5120b5d96 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:42.436130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da5312-4315-4519-a5c4-7ee5120b5d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:42.436561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da5312-4315-4519-a5c4-7ee5120b5d96 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:42.437591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da5312-4315-4519-a5c4-7ee5120b5d96 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:42.437949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da5312-4315-4519-a5c4-7ee5120b5d96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:42.439723 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da5312-4315-4519-a5c4-7ee5120b5d96 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:42.466197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 267/528] 10.222.0.64 () {56 vars in 1139 bytes} [Mon May 23 15:21:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:42.467339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da5312-4315-4519-a5c4-7ee5120b5d96 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:42.518551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f413b75-0692-41f5-bb2d-c8812259b18b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:42.558618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93aa76-1237-44e3-b03b-61436e18f9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:42.559430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93aa76-1237-44e3-b03b-61436e18f9da None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:42.559715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93aa76-1237-44e3-b03b-61436e18f9da None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:42.559995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93aa76-1237-44e3-b03b-61436e18f9da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:42.567832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93aa76-1237-44e3-b03b-61436e18f9da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:42.569394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93aa76-1237-44e3-b03b-61436e18f9da None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:42.584984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-8b93aa76-1237-44e3-b03b-61436e18f9da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:42.584984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:42.584984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:42.584984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:42.584984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:42.584984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:42.584984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:42.584984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:42.584984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:42.584984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:42.584984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:42.584984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:42.584984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:21:42.584984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:21:42.584984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:42.585913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:42.587599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:21:42.587599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:21:42.587599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:21:42.587599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:21:42.587599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:21:42.587599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:21:42.587599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:42.587599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:42.587599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 262/529] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:42.587599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93aa76-1237-44e3-b03b-61436e18f9da None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:42.588312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da5312-4315-4519-a5c4-7ee5120b5d96 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:42.591693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:42.633844 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e05cd9-3395-48b0-a1f4-06b0e676ff0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:42.635039 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e05cd9-3395-48b0-a1f4-06b0e676ff0f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:42.635479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e05cd9-3395-48b0-a1f4-06b0e676ff0f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:42.635908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e05cd9-3395-48b0-a1f4-06b0e676ff0f None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:42.637346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e05cd9-3395-48b0-a1f4-06b0e676ff0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:42.637835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e05cd9-3395-48b0-a1f4-06b0e676ff0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:42.639797 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e05cd9-3395-48b0-a1f4-06b0e676ff0f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:42.655953 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 268/530] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:42.657157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e05cd9-3395-48b0-a1f4-06b0e676ff0f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:42.685520 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:42.715541 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93aa76-1237-44e3-b03b-61436e18f9da None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:42.755750 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f33285-d0f3-4447-86b5-7cba9840787e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:42.756533 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f33285-d0f3-4447-86b5-7cba9840787e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:42.756827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f33285-d0f3-4447-86b5-7cba9840787e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:42.757137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f33285-d0f3-4447-86b5-7cba9840787e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:42.776647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e05cd9-3395-48b0-a1f4-06b0e676ff0f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:42.812065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac6e79-1a4b-42c2-ab80-2dc1cfd51923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:42.812973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac6e79-1a4b-42c2-ab80-2dc1cfd51923 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:42.813446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac6e79-1a4b-42c2-ab80-2dc1cfd51923 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:42.813866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac6e79-1a4b-42c2-ab80-2dc1cfd51923 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:42.814815 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac6e79-1a4b-42c2-ab80-2dc1cfd51923 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:42.815271 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac6e79-1a4b-42c2-ab80-2dc1cfd51923 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:42.817732 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac6e79-1a4b-42c2-ab80-2dc1cfd51923 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:42.834326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f33285-d0f3-4447-86b5-7cba9840787e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:42.835918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f33285-d0f3-4447-86b5-7cba9840787e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:42.839056 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:42.841316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 269/531] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:21:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:42.842243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac6e79-1a4b-42c2-ab80-2dc1cfd51923 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:42.936456 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:42.954066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 263/532] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:42.955241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f33285-d0f3-4447-86b5-7cba9840787e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:42.961243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac6e79-1a4b-42c2-ab80-2dc1cfd51923 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:43.009805 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208c66d-0d5d-4db7-a58d-5089d7ac0475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:43.010882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208c66d-0d5d-4db7-a58d-5089d7ac0475 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:43.011338 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208c66d-0d5d-4db7-a58d-5089d7ac0475 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:43.011765 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208c66d-0d5d-4db7-a58d-5089d7ac0475 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:43.019286 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208c66d-0d5d-4db7-a58d-5089d7ac0475 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:43.021897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208c66d-0d5d-4db7-a58d-5089d7ac0475 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:43.037026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-0208c66d-0d5d-4db7-a58d-5089d7ac0475 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:43.037026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:43.037026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:43.037026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:43.037026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:43.037026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:43.037026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:43.037026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:43.037026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:43.037026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:43.037026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:43.037026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:43.037026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:21:43.037026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:21:43.037026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:43.038164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:43.039107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:21:43.039107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:21:43.039107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:21:43.039107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:21:43.039107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:21:43.039107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:21:43.039107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:43.039107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:43.045453 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 270/533] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:43.046731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208c66d-0d5d-4db7-a58d-5089d7ac0475 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:43.071528 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f33285-d0f3-4447-86b5-7cba9840787e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:43.074577 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:43.119920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c666ce-ab5b-496c-a00d-307d97abd23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:43.120831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c666ce-ab5b-496c-a00d-307d97abd23c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:43.121149 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c666ce-ab5b-496c-a00d-307d97abd23c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:43.121433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c666ce-ab5b-496c-a00d-307d97abd23c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:43.122856 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c666ce-ab5b-496c-a00d-307d97abd23c None admin] RBAC: Adding query filter params (role.id=3d6e986dd2264e3cb7e01e25dbac6f97, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=45ee7a6c342548f98e7dc73f4f71a66d) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:43.123183 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c666ce-ab5b-496c-a00d-307d97abd23c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:43.124934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c666ce-ab5b-496c-a00d-307d97abd23c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:43.139774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 264/534] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:21:42 2022] GET /identity/v3/role_assignments?user.id=45ee7a6c342548f98e7dc73f4f71a66d&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=3d6e986dd2264e3cb7e01e25dbac6f97 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:43.141468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c666ce-ab5b-496c-a00d-307d97abd23c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:43.184611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208c66d-0d5d-4db7-a58d-5089d7ac0475 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:43.262153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f721906-3604-4170-9ce0-aafa26d4f0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:43.264930 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f721906-3604-4170-9ce0-aafa26d4f0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:43.265703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f721906-3604-4170-9ce0-aafa26d4f0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:43.266957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f721906-3604-4170-9ce0-aafa26d4f0c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:43.285453 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:43.292440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f721906-3604-4170-9ce0-aafa26d4f0c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:43.295565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f721906-3604-4170-9ce0-aafa26d4f0c0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:43.331639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-7f721906-3604-4170-9ce0-aafa26d4f0c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:43.331639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:43.331639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:43.331639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:43.331639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:43.331639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:43.331639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:43.331639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:43.331639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:43.331639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:43.331639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:43.331639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:43.331639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:43.331639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:43.331639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:43.333588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:43.334457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:43.334457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:43.334457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:43.334457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:43.334457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 271/535] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:43.335465 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f721906-3604-4170-9ce0-aafa26d4f0c0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:43.376377 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c666ce-ab5b-496c-a00d-307d97abd23c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:43.418833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02817c-07a9-427f-9708-a82bef60452c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:43.420304 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02817c-07a9-427f-9708-a82bef60452c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:43.420607 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02817c-07a9-427f-9708-a82bef60452c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:43.420890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02817c-07a9-427f-9708-a82bef60452c None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:43.421723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02817c-07a9-427f-9708-a82bef60452c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:43.422037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02817c-07a9-427f-9708-a82bef60452c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:43.423500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02817c-07a9-427f-9708-a82bef60452c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:43.438465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 265/536] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:21:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:43.445333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02817c-07a9-427f-9708-a82bef60452c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:43.494410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:43.547756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f721906-3604-4170-9ce0-aafa26d4f0c0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:43.618499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80dbbd-15f1-4bc0-99c7-7e38bc359f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:43.619180 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80dbbd-15f1-4bc0-99c7-7e38bc359f25 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:43.619468 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80dbbd-15f1-4bc0-99c7-7e38bc359f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:43.619750 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80dbbd-15f1-4bc0-99c7-7e38bc359f25 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:43.620490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e80dbbd-15f1-4bc0-99c7-7e38bc359f25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:43.620798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e80dbbd-15f1-4bc0-99c7-7e38bc359f25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:43.622190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e80dbbd-15f1-4bc0-99c7-7e38bc359f25 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:43.643589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02817c-07a9-427f-9708-a82bef60452c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:43.648205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 272/537] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:43.653855 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80dbbd-15f1-4bc0-99c7-7e38bc359f25 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:43.725824 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f9b0e-7a53-4382-9085-61c01341cce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:43.726517 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f9b0e-7a53-4382-9085-61c01341cce0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:43.726809 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f9b0e-7a53-4382-9085-61c01341cce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:43.727002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:43.727107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f9b0e-7a53-4382-9085-61c01341cce0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:43.727936 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f9b0e-7a53-4382-9085-61c01341cce0 None admin] RBAC: Adding query filter params (role.id=3d6e986dd2264e3cb7e01e25dbac6f97, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=8750b0450bf343f590c52b7d5c68d5df) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:43.728256 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f9b0e-7a53-4382-9085-61c01341cce0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:43.729777 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f9b0e-7a53-4382-9085-61c01341cce0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:43.747118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 266/538] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:21:43 2022] GET /identity/v3/role_assignments?user.id=8750b0450bf343f590c52b7d5c68d5df&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=3d6e986dd2264e3cb7e01e25dbac6f97 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:43.796455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80dbbd-15f1-4bc0-99c7-7e38bc359f25 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:43.846492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25c151-d2b4-422a-ae33-46c88e76ab50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:43.847243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25c151-d2b4-422a-ae33-46c88e76ab50 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:43.848072 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25c151-d2b4-422a-ae33-46c88e76ab50 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:43.848384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25c151-d2b4-422a-ae33-46c88e76ab50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:43.860765 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25c151-d2b4-422a-ae33-46c88e76ab50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:43.862299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25c151-d2b4-422a-ae33-46c88e76ab50 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:43.880298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-0b25c151-d2b4-422a-ae33-46c88e76ab50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:43.880298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:43.880298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:43.880298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:43.880298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:43.880298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:43.880298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:43.880298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:43.880298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:43.880298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:43.880298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:43.880298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:43.880298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:21:43.880298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:21:43.880298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:43.881267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:43.882146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:21:43.882146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:21:43.882146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:21:43.882146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:21:43.882146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:21:43.882146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:21:43.882146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:43.882146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:43.890753 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 273/539] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:43.896376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f9b0e-7a53-4382-9085-61c01341cce0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:44.071824 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:44.083465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f9b0e-7a53-4382-9085-61c01341cce0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:44.126407 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc6698-63e6-431f-9a92-b8e5a3cec23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:44.127008 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc6698-63e6-431f-9a92-b8e5a3cec23f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:44.127291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc6698-63e6-431f-9a92-b8e5a3cec23f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:44.127568 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc6698-63e6-431f-9a92-b8e5a3cec23f None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:44.128360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc6698-63e6-431f-9a92-b8e5a3cec23f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:44.128700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc6698-63e6-431f-9a92-b8e5a3cec23f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:44.130711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc6698-63e6-431f-9a92-b8e5a3cec23f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:44.147783 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 267/540] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:21:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:44.158201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25c151-d2b4-422a-ae33-46c88e76ab50 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:44.275289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25c151-d2b4-422a-ae33-46c88e76ab50 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:44.276740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0358b-99f5-48de-b3f2-036a1a788edc None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:44.277040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0358b-99f5-48de-b3f2-036a1a788edc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:44.277341 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0358b-99f5-48de-b3f2-036a1a788edc None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:44.279932 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 268/541] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:44.284303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73356cdf-a2a5-4238-a55d-5f994b9f8a41 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:44.284704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73356cdf-a2a5-4238-a55d-5f994b9f8a41 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:44.285635 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73356cdf-a2a5-4238-a55d-5f994b9f8a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:44.290612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:44.321421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-73356cdf-a2a5-4238-a55d-5f994b9f8a41 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:44.322644 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-73356cdf-a2a5-4238-a55d-5f994b9f8a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:44.334244 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1824a-6246-4973-acb0-d0d9bbedc5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:44.334920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1824a-6246-4973-acb0-d0d9bbedc5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:44.335212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1824a-6246-4973-acb0-d0d9bbedc5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:44.335494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1824a-6246-4973-acb0-d0d9bbedc5c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:44.336336 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1824a-6246-4973-acb0-d0d9bbedc5c8 None admin] RBAC: Adding query filter params (role.id=3d6e986dd2264e3cb7e01e25dbac6f97, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=12ceb8a355a44b9fb29918483379cb38) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:44.336652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1824a-6246-4973-acb0-d0d9bbedc5c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:44.338132 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1824a-6246-4973-acb0-d0d9bbedc5c8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:44.344694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 274/542] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:21:44 2022] GET /identity/v3/role_assignments?user.id=12ceb8a355a44b9fb29918483379cb38&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=3d6e986dd2264e3cb7e01e25dbac6f97 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:44.435840 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 269/543] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:44.472021 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1824a-6246-4973-acb0-d0d9bbedc5c8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:44.604217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1824a-6246-4973-acb0-d0d9bbedc5c8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:44.639787 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d3dc5-b306-423e-a25e-840bbc2e9eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:44.640960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d3dc5-b306-423e-a25e-840bbc2e9eed None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:44.641456 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d3dc5-b306-423e-a25e-840bbc2e9eed None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:44.641935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d3dc5-b306-423e-a25e-840bbc2e9eed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:44.654113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d3dc5-b306-423e-a25e-840bbc2e9eed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-bd7d3dc5-b306-423e-a25e-840bbc2e9eed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:44.662800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:44.663798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:44.664622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:44.664622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:44.664622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:44.664622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:44.664622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:44.664622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:44.664622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:44.664622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:44.664622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:44.664622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:44.664622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:44.664622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:44.667332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 275/544] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:44.674244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73356cdf-a2a5-4238-a55d-5f994b9f8a41 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:44.789431 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:44.803896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73356cdf-a2a5-4238-a55d-5f994b9f8a41 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:44.856636 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12614164-ff33-4f22-af89-f7bce119bd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:44.857362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12614164-ff33-4f22-af89-f7bce119bd12 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:44.857648 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12614164-ff33-4f22-af89-f7bce119bd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:44.857924 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12614164-ff33-4f22-af89-f7bce119bd12 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:44.858684 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12614164-ff33-4f22-af89-f7bce119bd12 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:44.858990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12614164-ff33-4f22-af89-f7bce119bd12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:44.860374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12614164-ff33-4f22-af89-f7bce119bd12 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:44.870022 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 270/545] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:21:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:44.875653 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d3dc5-b306-423e-a25e-840bbc2e9eed None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:44.937412 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:45.030271 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc8405-4ad1-41e0-805b-ca9842a276d9 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:45.030599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc8405-4ad1-41e0-805b-ca9842a276d9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:45.030843 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc8405-4ad1-41e0-805b-ca9842a276d9 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:45.032608 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d3dc5-b306-423e-a25e-840bbc2e9eed None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:45.033181 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 271/546] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:45.042497 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c61a9a-18d4-478b-9e16-1e77bab678ca None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:45.042790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c61a9a-18d4-478b-9e16-1e77bab678ca None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:45.043072 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c61a9a-18d4-478b-9e16-1e77bab678ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:45.073091 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a7c61a9a-18d4-478b-9e16-1e77bab678ca None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:45.074444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a7c61a9a-18d4-478b-9e16-1e77bab678ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:45.085627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea6edc-e016-417c-8a55-ceb59785d0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:45.086452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea6edc-e016-417c-8a55-ceb59785d0af None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:45.086735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea6edc-e016-417c-8a55-ceb59785d0af None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:45.087012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea6edc-e016-417c-8a55-ceb59785d0af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:45.104174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea6edc-e016-417c-8a55-ceb59785d0af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:45.105806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea6edc-e016-417c-8a55-ceb59785d0af None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:45.137619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f8ea6edc-e016-417c-8a55-ceb59785d0af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:45.137619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:45.137619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:45.137619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:45.137619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:45.137619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:45.137619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:45.137619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:45.137619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:45.137619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:45.137619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:45.137619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:45.137619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:45.137619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:45.137619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:45.138590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:45.139981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:45.139981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:45.139981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:45.139981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:45.139981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 276/547] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:45.145278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea6edc-e016-417c-8a55-ceb59785d0af None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:45.174589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 272/548] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:21:45.198052 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c61a9a-18d4-478b-9e16-1e77bab678ca None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:45.334222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:45.345467 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:45.355922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea6edc-e016-417c-8a55-ceb59785d0af None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:45.397273 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c89a2-4b3d-42c9-be71-906738948e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:45.398412 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c89a2-4b3d-42c9-be71-906738948e67 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:45.399969 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c89a2-4b3d-42c9-be71-906738948e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:45.400540 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c89a2-4b3d-42c9-be71-906738948e67 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:45.401499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c89a2-4b3d-42c9-be71-906738948e67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:45.402548 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c89a2-4b3d-42c9-be71-906738948e67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:45.404346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c89a2-4b3d-42c9-be71-906738948e67 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:45.417475 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c61a9a-18d4-478b-9e16-1e77bab678ca None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:45.430867 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 277/549] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:45.438887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c89a2-4b3d-42c9-be71-906738948e67 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:45.484555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a345ec-71ef-4407-809e-ec064d635d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:45.485396 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a345ec-71ef-4407-809e-ec064d635d4e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:45.485702 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a345ec-71ef-4407-809e-ec064d635d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:45.485971 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a345ec-71ef-4407-809e-ec064d635d4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:45.497035 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a345ec-71ef-4407-809e-ec064d635d4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-37a345ec-71ef-4407-809e-ec064d635d4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:45.505906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:45.506855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:45.507686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:45.507686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:45.507686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:45.507686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:45.507686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:45.507686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:45.507686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:45.507686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:45.507686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:45.507686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:45.507686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:45.507686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:45.507686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 273/550] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:45.513107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a345ec-71ef-4407-809e-ec064d635d4e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:45.613926 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c89a2-4b3d-42c9-be71-906738948e67 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:45.652237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5524f9-aed4-4710-8407-1d29453fa681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:45.653042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5524f9-aed4-4710-8407-1d29453fa681 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:45.653361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5524f9-aed4-4710-8407-1d29453fa681 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:45.653645 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5524f9-aed4-4710-8407-1d29453fa681 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:45.662815 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5524f9-aed4-4710-8407-1d29453fa681 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:45.665098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5524f9-aed4-4710-8407-1d29453fa681 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:45.667312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a345ec-71ef-4407-809e-ec064d635d4e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-0f5524f9-aed4-4710-8407-1d29453fa681 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:45.685093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:45.686069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:45.687013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:45.687013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:45.687013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:45.687013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:45.687013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:45.687013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:45.687013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:45.687013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:45.687013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:21:45.687013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:45.687013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 278/551] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:21:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:45.689497 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5524f9-aed4-4710-8407-1d29453fa681 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:45.709301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:45.774364 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122175b-0810-4030-9ab0-dfa6523c8be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:45.775228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122175b-0810-4030-9ab0-dfa6523c8be9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:45.775621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122175b-0810-4030-9ab0-dfa6523c8be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:45.776007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122175b-0810-4030-9ab0-dfa6523c8be9 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:45.776882 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122175b-0810-4030-9ab0-dfa6523c8be9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:45.777325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122175b-0810-4030-9ab0-dfa6523c8be9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:45.778820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122175b-0810-4030-9ab0-dfa6523c8be9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:45.780800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:45.800085 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 274/552] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:21:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:45.807931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122175b-0810-4030-9ab0-dfa6523c8be9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:45.828801 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5524f9-aed4-4710-8407-1d29453fa681 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:45.869913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bd981-6f1b-4101-ad80-b3df82128603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:45.871014 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bd981-6f1b-4101-ad80-b3df82128603 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:45.871486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bd981-6f1b-4101-ad80-b3df82128603 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:45.871970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bd981-6f1b-4101-ad80-b3df82128603 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:45.873217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bd981-6f1b-4101-ad80-b3df82128603 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:45.873785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bd981-6f1b-4101-ad80-b3df82128603 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:45.875852 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bd981-6f1b-4101-ad80-b3df82128603 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:45.897633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 279/553] 10.222.0.64 () {56 vars in 1135 bytes} [Mon May 23 15:21:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:45.903467 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bd981-6f1b-4101-ad80-b3df82128603 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:45.980740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:46.007800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122175b-0810-4030-9ab0-dfa6523c8be9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:46.020847 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bd981-6f1b-4101-ad80-b3df82128603 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:46.058127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e572f-84b4-4ec4-a9c3-a7a2d579e8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:46.059224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e572f-84b4-4ec4-a9c3-a7a2d579e8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:46.059521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e572f-84b4-4ec4-a9c3-a7a2d579e8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:46.059808 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e572f-84b4-4ec4-a9c3-a7a2d579e8b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:46.063542 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a453cd-9972-4825-aa8b-57607356dee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:46.064424 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a453cd-9972-4825-aa8b-57607356dee6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:46.064926 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a453cd-9972-4825-aa8b-57607356dee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:46.065491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a453cd-9972-4825-aa8b-57607356dee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:46.072366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e572f-84b4-4ec4-a9c3-a7a2d579e8b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:46.073926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e572f-84b4-4ec4-a9c3-a7a2d579e8b4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:46.098922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-1c0e572f-84b4-4ec4-a9c3-a7a2d579e8b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:46.098922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:46.098922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:46.098922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:46.098922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:46.098922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:46.098922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:46.098922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:46.098922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:46.098922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:46.098922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:46.098922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:46.098922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:46.098922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:46.098922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:46.099994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:46.100972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:46.100972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:46.100972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:46.100972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:46.105211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 275/554] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:21:46.107464 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8b8ed-b44f-47fc-87b0-55ce8fb78f5a None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:46.108113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8b8ed-b44f-47fc-87b0-55ce8fb78f5a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:46.108647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8b8ed-b44f-47fc-87b0-55ce8fb78f5a None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:46.114840 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 276/555] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:46.114840 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8b8ed-b44f-47fc-87b0-55ce8fb78f5a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:46.151176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a453cd-9972-4825-aa8b-57607356dee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:46.152574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a453cd-9972-4825-aa8b-57607356dee6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:46.184169 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:46.230375 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:46.263268 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8b8ed-b44f-47fc-87b0-55ce8fb78f5a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:46.276712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 280/556] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:21:46.278060 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036c74e-03e6-4464-8080-44b2f88a7689 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:46.278427 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036c74e-03e6-4464-8080-44b2f88a7689 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:46.278790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036c74e-03e6-4464-8080-44b2f88a7689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:46.296998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7036c74e-03e6-4464-8080-44b2f88a7689 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:46.298401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7036c74e-03e6-4464-8080-44b2f88a7689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:46.305927 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b04e1f-9779-484d-9509-ba77a774ec27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:46.307352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b04e1f-9779-484d-9509-ba77a774ec27 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:46.307844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b04e1f-9779-484d-9509-ba77a774ec27 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:46.308302 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b04e1f-9779-484d-9509-ba77a774ec27 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:46.309697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b04e1f-9779-484d-9509-ba77a774ec27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:46.310191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b04e1f-9779-484d-9509-ba77a774ec27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:46.312478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b04e1f-9779-484d-9509-ba77a774ec27 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:46.329010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 277/557] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:46.330130 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b04e1f-9779-484d-9509-ba77a774ec27 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:46.390213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 281/558] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:46.391966 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036c74e-03e6-4464-8080-44b2f88a7689 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:46.446756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b04e1f-9779-484d-9509-ba77a774ec27 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:46.457209 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:46.487725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76515fca-c715-4edf-b644-aeb755593528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:46.488842 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76515fca-c715-4edf-b644-aeb755593528 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:46.489174 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76515fca-c715-4edf-b644-aeb755593528 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:46.489457 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76515fca-c715-4edf-b644-aeb755593528 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:46.499245 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:46.501329 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76515fca-c715-4edf-b644-aeb755593528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:46.502814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76515fca-c715-4edf-b644-aeb755593528 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:46.505814 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036c74e-03e6-4464-8080-44b2f88a7689 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:46.521210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-76515fca-c715-4edf-b644-aeb755593528 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:46.521210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:46.521210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:46.521210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:46.521210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:46.521210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:46.521210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:46.521210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:46.521210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:46.521210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:46.521210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:46.521210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:46.521210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:46.521210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:46.521210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:46.522322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:46.523252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:46.523252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:46.523252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:46.523252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:46.523252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 278/559] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:46.523737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f3bbf-fe6a-4a0a-ae1c-29f7647d09c7 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:46.524003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f3bbf-fe6a-4a0a-ae1c-29f7647d09c7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:46.524280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f3bbf-fe6a-4a0a-ae1c-29f7647d09c7 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:46.529672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 279/560] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:46.529672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f3bbf-fe6a-4a0a-ae1c-29f7647d09c7 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:46.553751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a588325-1451-4b76-a7cd-7fb52be03b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:46.554557 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a588325-1451-4b76-a7cd-7fb52be03b21 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:46.554872 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a588325-1451-4b76-a7cd-7fb52be03b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:46.555130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a588325-1451-4b76-a7cd-7fb52be03b21 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:46.562436 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a588325-1451-4b76-a7cd-7fb52be03b21 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:46.563890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a588325-1451-4b76-a7cd-7fb52be03b21 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6a588325-1451-4b76-a7cd-7fb52be03b21 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:46.579278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:46.580371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:46.581393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:46.581393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:46.581393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:46.581393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:46.581393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:46.581393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:46.581393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:46.581393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:46.581393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:21:46.581393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:46.581393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 282/561] 10.222.0.64 () {56 vars in 1093 bytes} [Mon May 23 15:21:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:46.581393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a588325-1451-4b76-a7cd-7fb52be03b21 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:46.648090 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f3bbf-fe6a-4a0a-ae1c-29f7647d09c7 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:46.685216 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d4ea3-6bdb-42ac-b309-efee82e7ae94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:46.686045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d4ea3-6bdb-42ac-b309-efee82e7ae94 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:46.686337 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d4ea3-6bdb-42ac-b309-efee82e7ae94 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:46.686630 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d4ea3-6bdb-42ac-b309-efee82e7ae94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:46.690186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a588325-1451-4b76-a7cd-7fb52be03b21 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:46.706070 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d4ea3-6bdb-42ac-b309-efee82e7ae94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-a89d4ea3-6bdb-42ac-b309-efee82e7ae94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:46.714964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:46.716078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:46.717046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:46.717046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:46.717046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:46.717046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:46.717046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:46.717046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:46.717046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:46.717046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:46.717046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:46.717046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:46.717046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:46.717046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:46.717046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 280/562] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:46.717808 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931109b9-b3d1-4e3d-afa0-256e83fde26a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:46.719042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931109b9-b3d1-4e3d-afa0-256e83fde26a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:46.719042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931109b9-b3d1-4e3d-afa0-256e83fde26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:46.736889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbaff5-d522-47cc-a81b-cbfa90be664c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:46.737560 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-931109b9-b3d1-4e3d-afa0-256e83fde26a None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:46.737666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbaff5-d522-47cc-a81b-cbfa90be664c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:46.737918 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbaff5-d522-47cc-a81b-cbfa90be664c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:46.738202 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbaff5-d522-47cc-a81b-cbfa90be664c None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:46.739105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbaff5-d522-47cc-a81b-cbfa90be664c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:46.739417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbaff5-d522-47cc-a81b-cbfa90be664c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:46.739885 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-931109b9-b3d1-4e3d-afa0-256e83fde26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:46.740835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbaff5-d522-47cc-a81b-cbfa90be664c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:46.758527 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 283/563] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:46.758527 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbaff5-d522-47cc-a81b-cbfa90be664c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:46.762354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:46.774187 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:46.817191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 281/564] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:46.821306 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931109b9-b3d1-4e3d-afa0-256e83fde26a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:46.873281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbaff5-d522-47cc-a81b-cbfa90be664c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:46.908423 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5958bef-49b5-4b3e-bfdf-7c2a963c9f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:46.909434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5958bef-49b5-4b3e-bfdf-7c2a963c9f2d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:46.910438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5958bef-49b5-4b3e-bfdf-7c2a963c9f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:46.910860 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5958bef-49b5-4b3e-bfdf-7c2a963c9f2d None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:46.911986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5958bef-49b5-4b3e-bfdf-7c2a963c9f2d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:46.912471 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5958bef-49b5-4b3e-bfdf-7c2a963c9f2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:46.914157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5958bef-49b5-4b3e-bfdf-7c2a963c9f2d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:46.930839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931109b9-b3d1-4e3d-afa0-256e83fde26a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:46.931686 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 284/565] 10.222.0.64 () {56 vars in 1133 bytes} [Mon May 23 15:21:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:46.932489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5958bef-49b5-4b3e-bfdf-7c2a963c9f2d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:46.966723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbcf71e-fdca-4e85-91d5-f0f72d4874e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:46.967304 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcf71e-fdca-4e85-91d5-f0f72d4874e9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:46.967588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcf71e-fdca-4e85-91d5-f0f72d4874e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:46.967863 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcf71e-fdca-4e85-91d5-f0f72d4874e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:46.968605 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcf71e-fdca-4e85-91d5-f0f72d4874e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:46.968915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcf71e-fdca-4e85-91d5-f0f72d4874e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:46.970319 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcf71e-fdca-4e85-91d5-f0f72d4874e9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:46.978087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 282/566] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:21:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:46.978861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbcf71e-fdca-4e85-91d5-f0f72d4874e9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:46.993007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:47.006478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:47.030383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5958bef-49b5-4b3e-bfdf-7c2a963c9f2d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:47.064302 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578b25b-01d4-4eea-a71e-bd2d89e2cdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:47.065119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578b25b-01d4-4eea-a71e-bd2d89e2cdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.065406 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578b25b-01d4-4eea-a71e-bd2d89e2cdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.065685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578b25b-01d4-4eea-a71e-bd2d89e2cdc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.072583 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578b25b-01d4-4eea-a71e-bd2d89e2cdc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:47.074042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578b25b-01d4-4eea-a71e-bd2d89e2cdc6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:47.076456 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbcf71e-fdca-4e85-91d5-f0f72d4874e9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:47.088284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-8578b25b-01d4-4eea-a71e-bd2d89e2cdc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:47.088284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:47.088284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:47.088284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:47.088284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:47.088284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:47.088284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:47.088284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:47.088284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:47.088284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:47.088284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:47.088284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:47.088284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:21:47.088284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:21:47.088284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:47.089232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:47.090144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:21:47.090144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:21:47.090144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:21:47.090144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:21:47.090144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:21:47.090144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:21:47.090144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:47.090144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:47.090144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 285/567] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:47.090685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a839e-4e26-418a-bc14-90da89a5793a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.090994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a839e-4e26-418a-bc14-90da89a5793a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.091244 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a839e-4e26-418a-bc14-90da89a5793a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.092498 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 286/568] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:47.093218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a839e-4e26-418a-bc14-90da89a5793a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:47.114580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b463af-85e6-4b92-aa1f-b2724c88e6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:47.115381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b463af-85e6-4b92-aa1f-b2724c88e6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.115676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b463af-85e6-4b92-aa1f-b2724c88e6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.115955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b463af-85e6-4b92-aa1f-b2724c88e6a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.126922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b463af-85e6-4b92-aa1f-b2724c88e6a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-62b463af-85e6-4b92-aa1f-b2724c88e6a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:47.135237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:47.136191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:47.137045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:47.137045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:47.137045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:47.137045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:47.137045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:47.137045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:47.137045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:47.137045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:47.137045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:47.137045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:47.137045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:47.137045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:47.137045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 283/569] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:47.138353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b463af-85e6-4b92-aa1f-b2724c88e6a2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:47.196902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a839e-4e26-418a-bc14-90da89a5793a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:47.207996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:47.230913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fac79a-eea5-4ab9-95d3-ac671ff73a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:47.231650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fac79a-eea5-4ab9-95d3-ac671ff73a6a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.231943 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fac79a-eea5-4ab9-95d3-ac671ff73a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.232225 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fac79a-eea5-4ab9-95d3-ac671ff73a6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.234155 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:47.237171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b463af-85e6-4b92-aa1f-b2724c88e6a2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:47.242920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fac79a-eea5-4ab9-95d3-ac671ff73a6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:47.244360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fac79a-eea5-4ab9-95d3-ac671ff73a6a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:47.261448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-89fac79a-eea5-4ab9-95d3-ac671ff73a6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:47.261448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:47.261448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:47.261448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:47.261448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:47.261448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:47.261448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:47.261448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:47.261448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:47.261448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:47.261448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:47.261448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:47.261448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:47.261448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:47.261448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:47.262432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:47.263350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:47.263350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:47.263350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:47.263350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:47.263350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 287/570] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:47.263650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fac79a-eea5-4ab9-95d3-ac671ff73a6a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:47.279239 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e6e1e-b5c1-4dd0-8e81-6f2a3489a786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:47.280032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e6e1e-b5c1-4dd0-8e81-6f2a3489a786 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.280319 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e6e1e-b5c1-4dd0-8e81-6f2a3489a786 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.280597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e6e1e-b5c1-4dd0-8e81-6f2a3489a786 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.292220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e6e1e-b5c1-4dd0-8e81-6f2a3489a786 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:47.293732 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e6e1e-b5c1-4dd0-8e81-6f2a3489a786 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:47.310604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-d84e6e1e-b5c1-4dd0-8e81-6f2a3489a786 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:47.310604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:47.310604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:47.310604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:47.310604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:47.310604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:47.310604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:47.310604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:47.310604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:47.310604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:47.310604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:47.310604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:47.310604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:47.310604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:47.310604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:47.311548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:47.312547 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:47.312547 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:47.312547 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:47.312547 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:47.312547 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 284/571] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:47.313048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a99e60-27c4-40fc-890a-88206c8d2364 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.313494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a99e60-27c4-40fc-890a-88206c8d2364 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.313494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a99e60-27c4-40fc-890a-88206c8d2364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.330245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-66a99e60-27c4-40fc-890a-88206c8d2364 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:47.331437 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-66a99e60-27c4-40fc-890a-88206c8d2364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:47.366186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fac79a-eea5-4ab9-95d3-ac671ff73a6a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:47.399304 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0508ec9-954f-41dc-bf0c-3b49d1f2415f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:47.399963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0508ec9-954f-41dc-bf0c-3b49d1f2415f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.400249 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0508ec9-954f-41dc-bf0c-3b49d1f2415f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.400374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 285/572] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:47.400528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0508ec9-954f-41dc-bf0c-3b49d1f2415f None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.401222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a99e60-27c4-40fc-890a-88206c8d2364 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:47.401347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0508ec9-954f-41dc-bf0c-3b49d1f2415f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:47.401660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0508ec9-954f-41dc-bf0c-3b49d1f2415f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:47.403078 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0508ec9-954f-41dc-bf0c-3b49d1f2415f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:47.415120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 288/573] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:21:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:47.415909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0508ec9-954f-41dc-bf0c-3b49d1f2415f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:47.465625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:47.470234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:47.510708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a99e60-27c4-40fc-890a-88206c8d2364 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:47.524839 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0508ec9-954f-41dc-bf0c-3b49d1f2415f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:47.543519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea298ac-30df-450b-8143-cee2bdf8a917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:47.544193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea298ac-30df-450b-8143-cee2bdf8a917 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.544473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea298ac-30df-450b-8143-cee2bdf8a917 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.544749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea298ac-30df-450b-8143-cee2bdf8a917 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.545529 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea298ac-30df-450b-8143-cee2bdf8a917 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:47.545841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea298ac-30df-450b-8143-cee2bdf8a917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:47.547214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea298ac-30df-450b-8143-cee2bdf8a917 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:47.560022 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 286/574] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:21:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:47.560022 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea298ac-30df-450b-8143-cee2bdf8a917 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:47.560255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff446ae6-9a98-4afb-ae2d-2f7274a05fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:47.560255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff446ae6-9a98-4afb-ae2d-2f7274a05fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.560255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff446ae6-9a98-4afb-ae2d-2f7274a05fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.560255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff446ae6-9a98-4afb-ae2d-2f7274a05fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.560532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff446ae6-9a98-4afb-ae2d-2f7274a05fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:47.560532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff446ae6-9a98-4afb-ae2d-2f7274a05fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:47.561835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff446ae6-9a98-4afb-ae2d-2f7274a05fb3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:47.573542 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 289/575] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:47.574360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff446ae6-9a98-4afb-ae2d-2f7274a05fb3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:47.671107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea298ac-30df-450b-8143-cee2bdf8a917 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:47.688915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:47.708939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff446ae6-9a98-4afb-ae2d-2f7274a05fb3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:47.714495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1289e-94cc-43de-b067-1aa6567daa8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:47.714495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1289e-94cc-43de-b067-1aa6567daa8e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.714801 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1289e-94cc-43de-b067-1aa6567daa8e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.714898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1289e-94cc-43de-b067-1aa6567daa8e None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.715566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1289e-94cc-43de-b067-1aa6567daa8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:47.715881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1289e-94cc-43de-b067-1aa6567daa8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:47.716453 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:47.717281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1289e-94cc-43de-b067-1aa6567daa8e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:47.730042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 287/576] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:47.730890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1289e-94cc-43de-b067-1aa6567daa8e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:47.745724 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cb4e3-929d-4352-99ac-b95591da7855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:47.746383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cb4e3-929d-4352-99ac-b95591da7855 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.746668 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cb4e3-929d-4352-99ac-b95591da7855 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.746946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cb4e3-929d-4352-99ac-b95591da7855 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.747783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cb4e3-929d-4352-99ac-b95591da7855 None admin] RBAC: Adding query filter params (role.id=3d6e986dd2264e3cb7e01e25dbac6f97, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=f8c79c36c7204368816593f0e473191b) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:47.748090 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cb4e3-929d-4352-99ac-b95591da7855 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:47.750354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cb4e3-929d-4352-99ac-b95591da7855 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:47.757387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 290/577] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:21:47 2022] GET /identity/v3/role_assignments?user.id=f8c79c36c7204368816593f0e473191b&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=3d6e986dd2264e3cb7e01e25dbac6f97 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:47.758204 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cb4e3-929d-4352-99ac-b95591da7855 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:47.873210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1289e-94cc-43de-b067-1aa6567daa8e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:47.900759 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cb4e3-929d-4352-99ac-b95591da7855 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:47.923302 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06667dd9-2356-4c1b-8162-c9c1f3eb5100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:47.924161 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06667dd9-2356-4c1b-8162-c9c1f3eb5100 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.924456 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06667dd9-2356-4c1b-8162-c9c1f3eb5100 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.924729 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06667dd9-2356-4c1b-8162-c9c1f3eb5100 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.946480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06667dd9-2356-4c1b-8162-c9c1f3eb5100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:47.947535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e71e8-aecb-4de9-85f2-3bb691317383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:47.948474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e71e8-aecb-4de9-85f2-3bb691317383 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:47.948763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e71e8-aecb-4de9-85f2-3bb691317383 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:47.949044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e71e8-aecb-4de9-85f2-3bb691317383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-06667dd9-2356-4c1b-8162-c9c1f3eb5100 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:47.959205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:47.960208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:47.961097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:47.961097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:47.961097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:47.961097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:47.961097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:47.961097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:47.961097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:47.961097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:47.961097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:47.961097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:47.961097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:47.961097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:47.961097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 288/578] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:47.961097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06667dd9-2356-4c1b-8162-c9c1f3eb5100 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:47.968085 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e71e8-aecb-4de9-85f2-3bb691317383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:47.969590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e71e8-aecb-4de9-85f2-3bb691317383 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:47.998623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b41e71e8-aecb-4de9-85f2-3bb691317383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:47.998623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:47.998623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:47.998623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:47.998623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:47.998623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:47.998623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:47.998623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:47.998623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:47.998623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:47.998623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:47.998623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:47.998623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:47.998623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:47.998623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:47.999596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:48.000509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:48.000509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:48.000509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:48.000509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:48.000736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 291/579] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:48.002036 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e71e8-aecb-4de9-85f2-3bb691317383 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:48.013038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:48.032958 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:48.091184 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06667dd9-2356-4c1b-8162-c9c1f3eb5100 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:48.133266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e71e8-aecb-4de9-85f2-3bb691317383 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:48.135043 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7078d1-f1a9-4cf6-a6c8-d3a0b6452d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:48.136104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7078d1-f1a9-4cf6-a6c8-d3a0b6452d2b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:48.136495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7078d1-f1a9-4cf6-a6c8-d3a0b6452d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:48.136815 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7078d1-f1a9-4cf6-a6c8-d3a0b6452d2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:48.146294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7078d1-f1a9-4cf6-a6c8-d3a0b6452d2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:48.147882 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7078d1-f1a9-4cf6-a6c8-d3a0b6452d2b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:48.166440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-ea7078d1-f1a9-4cf6-a6c8-d3a0b6452d2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:48.166440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:48.166440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:48.166440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:48.166440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:48.166440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:48.166440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:48.166440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:48.166440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:48.166440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:48.166440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:48.166440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:48.166440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:21:48.166440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:21:48.166440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:48.167523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:48.168492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:21:48.168492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:21:48.168492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:21:48.168492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:21:48.168492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:21:48.168492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:21:48.168492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:48.168492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:48.169382 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 289/580] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:48.169973 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7078d1-f1a9-4cf6-a6c8-d3a0b6452d2b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:48.181456 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477abd6b-e0a0-4da1-a22e-5a5a3bd08aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:48.182355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477abd6b-e0a0-4da1-a22e-5a5a3bd08aaa None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:48.182825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477abd6b-e0a0-4da1-a22e-5a5a3bd08aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:48.183322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477abd6b-e0a0-4da1-a22e-5a5a3bd08aaa None admin] PATH_INFO: `/v3/users/glance` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:48.194068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477abd6b-e0a0-4da1-a22e-5a5a3bd08aaa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:48.195596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477abd6b-e0a0-4da1-a22e-5a5a3bd08aaa None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-477abd6b-e0a0-4da1-a22e-5a5a3bd08aaa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:48.210475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:48.211690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:48.212691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:48.212691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:48.212691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:48.212691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:48.212691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:48.212691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:48.212691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:48.212691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:48.212691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 15:21:48.212691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:48.212691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 292/581] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:48.213983 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477abd6b-e0a0-4da1-a22e-5a5a3bd08aaa None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:48.305394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:48.313610 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7078d1-f1a9-4cf6-a6c8-d3a0b6452d2b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:48.318089 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:48.338409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477abd6b-e0a0-4da1-a22e-5a5a3bd08aaa None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:48.358728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3847e-82e3-4f14-b80e-acc5899d67ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:48.359723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3847e-82e3-4f14-b80e-acc5899d67ea None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:48.359723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3847e-82e3-4f14-b80e-acc5899d67ea None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:48.360237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3847e-82e3-4f14-b80e-acc5899d67ea None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:48.360698 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3847e-82e3-4f14-b80e-acc5899d67ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:48.361007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3847e-82e3-4f14-b80e-acc5899d67ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:48.362416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3847e-82e3-4f14-b80e-acc5899d67ea None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:48.381208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 290/582] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:21:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:48.382872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3847e-82e3-4f14-b80e-acc5899d67ea None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:48.386177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8bf7b-9c24-41c2-8d3e-ae62d27bf1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:48.386804 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8bf7b-9c24-41c2-8d3e-ae62d27bf1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:48.387084 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8bf7b-9c24-41c2-8d3e-ae62d27bf1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:48.388424 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8bf7b-9c24-41c2-8d3e-ae62d27bf1b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:48.389208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba8bf7b-9c24-41c2-8d3e-ae62d27bf1b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:48.389543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba8bf7b-9c24-41c2-8d3e-ae62d27bf1b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:48.390922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba8bf7b-9c24-41c2-8d3e-ae62d27bf1b3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:48.405866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 293/583] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:48.407494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8bf7b-9c24-41c2-8d3e-ae62d27bf1b3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:48.512806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3847e-82e3-4f14-b80e-acc5899d67ea None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:48.537277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8bf7b-9c24-41c2-8d3e-ae62d27bf1b3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:48.549751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f933e-5d8f-40ca-914f-95a58a24de00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:48.550446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f933e-5d8f-40ca-914f-95a58a24de00 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:48.550802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f933e-5d8f-40ca-914f-95a58a24de00 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:48.551000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f933e-5d8f-40ca-914f-95a58a24de00 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:48.551787 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f933e-5d8f-40ca-914f-95a58a24de00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:48.552100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f933e-5d8f-40ca-914f-95a58a24de00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:48.553898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f933e-5d8f-40ca-914f-95a58a24de00 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:48.571957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 291/584] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:21:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:48.571957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f933e-5d8f-40ca-914f-95a58a24de00 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:48.577590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0493be-61d8-47ad-93d4-7dd174da0e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:48.578666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0493be-61d8-47ad-93d4-7dd174da0e88 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:48.578666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0493be-61d8-47ad-93d4-7dd174da0e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:48.578853 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0493be-61d8-47ad-93d4-7dd174da0e88 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:48.579664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0493be-61d8-47ad-93d4-7dd174da0e88 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:48.579975 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0493be-61d8-47ad-93d4-7dd174da0e88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:48.581427 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0493be-61d8-47ad-93d4-7dd174da0e88 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:48.590741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:48.598204 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:48.605806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 294/585] 10.222.0.64 () {56 vars in 1137 bytes} [Mon May 23 15:21:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:48.605806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0493be-61d8-47ad-93d4-7dd174da0e88 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:48.688806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f933e-5d8f-40ca-914f-95a58a24de00 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:48.712932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0493be-61d8-47ad-93d4-7dd174da0e88 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:48.727763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cadf16-2752-4d68-8fc2-0f63e0afeeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:48.728579 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cadf16-2752-4d68-8fc2-0f63e0afeeaf None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:48.728868 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cadf16-2752-4d68-8fc2-0f63e0afeeaf None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:48.729183 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cadf16-2752-4d68-8fc2-0f63e0afeeaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:48.740858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cadf16-2752-4d68-8fc2-0f63e0afeeaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:48.742360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cadf16-2752-4d68-8fc2-0f63e0afeeaf None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:48.750757 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8522d33-077e-4780-a2b1-3a9e5de1a977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:48.751559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8522d33-077e-4780-a2b1-3a9e5de1a977 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:48.751847 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8522d33-077e-4780-a2b1-3a9e5de1a977 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:48.752130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8522d33-077e-4780-a2b1-3a9e5de1a977 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:48.760063 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8522d33-077e-4780-a2b1-3a9e5de1a977 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:48.760589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-73cadf16-2752-4d68-8fc2-0f63e0afeeaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:48.760589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:48.760589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:48.760589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:48.760589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:48.760589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:48.760589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:48.760589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:48.760589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:48.760589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:48.760589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:48.760589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:48.760589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:48.760589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:48.760589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:48.761509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:48.762419 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:48.762419 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:48.762419 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:48.762419 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:48.762643 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8522d33-077e-4780-a2b1-3a9e5de1a977 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:48.762714 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 292/586] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:48.763312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cadf16-2752-4d68-8fc2-0f63e0afeeaf None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b8522d33-077e-4780-a2b1-3a9e5de1a977 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:48.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:48.782756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:48.783695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:48.783695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:48.783695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:48.783695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:48.783695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:48.783695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:48.783695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:48.783695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:48.783695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:21:48.783695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:48.783695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 295/587] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:48.784321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8522d33-077e-4780-a2b1-3a9e5de1a977 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:48.845774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:48.851660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:48.888692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cadf16-2752-4d68-8fc2-0f63e0afeeaf None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:48.894093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8522d33-077e-4780-a2b1-3a9e5de1a977 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:48.932256 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d1f6a-8eec-4f09-834a-5b2a9a145e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:48.933180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d1f6a-8eec-4f09-834a-5b2a9a145e45 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:48.933475 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d1f6a-8eec-4f09-834a-5b2a9a145e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:48.933679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6429bd-2bf5-44b3-a1fd-3d86548eb8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:48.933803 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d1f6a-8eec-4f09-834a-5b2a9a145e45 None admin] PATH_INFO: `/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/8da96bed73794fee8f719ec904936164/roles/3d6e986dd2264e3cb7e01e25dbac6f97` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:48.934944 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6429bd-2bf5-44b3-a1fd-3d86548eb8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:48.935471 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6429bd-2bf5-44b3-a1fd-3d86548eb8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:48.935907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6429bd-2bf5-44b3-a1fd-3d86548eb8f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:48.949088 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6429bd-2bf5-44b3-a1fd-3d86548eb8f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:48.950070 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d1f6a-8eec-4f09-834a-5b2a9a145e45 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d81eafac474f37a71ee5cc292d172f, user_id=8da96bed73794fee8f719ec904936164, role_id=3d6e986dd2264e3cb7e01e25dbac6f97)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:48.950957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6429bd-2bf5-44b3-a1fd-3d86548eb8f9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:48.951531 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d1f6a-8eec-4f09-834a-5b2a9a145e45 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:48.970494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-ab6429bd-2bf5-44b3-a1fd-3d86548eb8f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:48.970494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:48.970494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:48.970494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:48.970494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:48.970494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:48.970494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:48.970494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:48.970494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:48.970494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:48.970494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:48.970494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:48.970494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:48.970494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:48.970494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:48.971629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:48.973309 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:48.973309 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:48.973309 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:48.973309 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:48.973309 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 296/588] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:48.974001 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6429bd-2bf5-44b3-a1fd-3d86548eb8f9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:49.005326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 293/589] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:21:48 2022] PUT /identity/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/8da96bed73794fee8f719ec904936164/roles/3d6e986dd2264e3cb7e01e25dbac6f97 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:21:49.006219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d1f6a-8eec-4f09-834a-5b2a9a145e45 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:49.092479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6429bd-2bf5-44b3-a1fd-3d86548eb8f9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:49.109385 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:49.136076 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db796b27-699c-44af-81e6-2bd141a3a4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:49.136990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db796b27-699c-44af-81e6-2bd141a3a4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:49.137403 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db796b27-699c-44af-81e6-2bd141a3a4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:49.137777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db796b27-699c-44af-81e6-2bd141a3a4f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:49.138679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db796b27-699c-44af-81e6-2bd141a3a4f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:49.139094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db796b27-699c-44af-81e6-2bd141a3a4f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:49.140701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db796b27-699c-44af-81e6-2bd141a3a4f7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:49.158649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 297/590] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:49.158649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db796b27-699c-44af-81e6-2bd141a3a4f7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:49.168689 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d1f6a-8eec-4f09-834a-5b2a9a145e45 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:49.192294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:49.211118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f23bb-2258-4160-8256-866f25778a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:49.211733 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f23bb-2258-4160-8256-866f25778a5a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:49.212016 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f23bb-2258-4160-8256-866f25778a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:49.212294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f23bb-2258-4160-8256-866f25778a5a None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:49.213797 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f23bb-2258-4160-8256-866f25778a5a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:49.214112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f23bb-2258-4160-8256-866f25778a5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:49.216010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f23bb-2258-4160-8256-866f25778a5a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:49.241187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 294/591] 10.222.0.64 () {56 vars in 1137 bytes} [Mon May 23 15:21:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:49.241187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f23bb-2258-4160-8256-866f25778a5a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:49.297337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db796b27-699c-44af-81e6-2bd141a3a4f7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:49.348833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489e86d-3597-4d31-923b-4e33ef9608f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:49.349695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489e86d-3597-4d31-923b-4e33ef9608f1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:49.349968 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f23bb-2258-4160-8256-866f25778a5a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:49.350243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489e86d-3597-4d31-923b-4e33ef9608f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:49.350598 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489e86d-3597-4d31-923b-4e33ef9608f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:49.351401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489e86d-3597-4d31-923b-4e33ef9608f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:49.351785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489e86d-3597-4d31-923b-4e33ef9608f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:49.354374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489e86d-3597-4d31-923b-4e33ef9608f1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:49.373147 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 298/592] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:49.374259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489e86d-3597-4d31-923b-4e33ef9608f1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:49.396807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e7bc0-ee86-4fe6-9425-7f092b8a7a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:49.398193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e7bc0-ee86-4fe6-9425-7f092b8a7a86 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:49.398656 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e7bc0-ee86-4fe6-9425-7f092b8a7a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:49.399171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e7bc0-ee86-4fe6-9425-7f092b8a7a86 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:49.407754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e7bc0-ee86-4fe6-9425-7f092b8a7a86 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:49.409552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e7bc0-ee86-4fe6-9425-7f092b8a7a86 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:49.417430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-605e7bc0-ee86-4fe6-9425-7f092b8a7a86 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:49.427049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:49.431814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:49.432794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:49.432794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:49.432794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:49.432794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:49.432794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:49.432794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:49.432794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:49.432794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:49.432794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:21:49.432794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:49.432794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 295/593] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:49.432794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e7bc0-ee86-4fe6-9425-7f092b8a7a86 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:49.476219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:49.512256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489e86d-3597-4d31-923b-4e33ef9608f1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:49.545552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e7bc0-ee86-4fe6-9425-7f092b8a7a86 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:49.562084 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9ed55-5605-4e37-ae96-e3fa4c18d485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:49.562927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9ed55-5605-4e37-ae96-e3fa4c18d485 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:49.563215 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9ed55-5605-4e37-ae96-e3fa4c18d485 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:49.563494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9ed55-5605-4e37-ae96-e3fa4c18d485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:49.575159 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9ed55-5605-4e37-ae96-e3fa4c18d485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:49.576733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9ed55-5605-4e37-ae96-e3fa4c18d485 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:49.585993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef1b71-cf4c-40b1-93cd-6c093f54f118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:49.586804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef1b71-cf4c-40b1-93cd-6c093f54f118 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:49.587088 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef1b71-cf4c-40b1-93cd-6c093f54f118 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:49.587365 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef1b71-cf4c-40b1-93cd-6c093f54f118 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:49.594668 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef1b71-cf4c-40b1-93cd-6c093f54f118 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:49.596123 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef1b71-cf4c-40b1-93cd-6c093f54f118 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:49.596337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-10e9ed55-5605-4e37-ae96-e3fa4c18d485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:49.596337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:49.596337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:49.596337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:49.596337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:49.596337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:49.596337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:49.596337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:49.596337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:49.596337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:49.596337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:49.596337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:49.596337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:49.596337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:49.596337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:49.597809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:49.598808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:49.598808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:49.598808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:49.598808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:49.598808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 299/594] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:49.599422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9ed55-5605-4e37-ae96-e3fa4c18d485 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:49.620003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-9fef1b71-cf4c-40b1-93cd-6c093f54f118 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:49.620003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:49.620003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:49.620003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:49.620003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:49.620003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:49.620003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:49.620003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:49.620003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:49.620003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:49.620003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:49.620003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:49.620003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:21:49.620003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:21:49.620003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:49.622469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:49.624675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:21:49.624675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:21:49.624675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:21:49.624675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:21:49.624675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:21:49.624675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:21:49.624675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:49.624675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:49.624675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 296/595] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:49.624675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef1b71-cf4c-40b1-93cd-6c093f54f118 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:49.823095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:49.842629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9ed55-5605-4e37-ae96-e3fa4c18d485 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:49.877414 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:49.879773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb7dbd-0ee2-407b-b427-d8fbaa63c4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:49.880658 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb7dbd-0ee2-407b-b427-d8fbaa63c4ba None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:49.881129 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb7dbd-0ee2-407b-b427-d8fbaa63c4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:49.881553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb7dbd-0ee2-407b-b427-d8fbaa63c4ba None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:49.882509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef1b71-cf4c-40b1-93cd-6c093f54f118 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:49.882709 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb7dbd-0ee2-407b-b427-d8fbaa63c4ba None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:49.883183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb7dbd-0ee2-407b-b427-d8fbaa63c4ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:49.884768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb7dbd-0ee2-407b-b427-d8fbaa63c4ba None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:49.905218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 300/596] 10.222.0.64 () {56 vars in 1139 bytes} [Mon May 23 15:21:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:49.906394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb7dbd-0ee2-407b-b427-d8fbaa63c4ba None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:49.923592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7a707-dca2-40f3-9663-c6c483bd748b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:49.924423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7a707-dca2-40f3-9663-c6c483bd748b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:49.924827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7a707-dca2-40f3-9663-c6c483bd748b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:49.925358 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7a707-dca2-40f3-9663-c6c483bd748b None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:49.926441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7a707-dca2-40f3-9663-c6c483bd748b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:49.926780 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7a707-dca2-40f3-9663-c6c483bd748b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:49.928522 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7a707-dca2-40f3-9663-c6c483bd748b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:49.945933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 297/597] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:49.947001 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7a707-dca2-40f3-9663-c6c483bd748b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:50.025261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb7dbd-0ee2-407b-b427-d8fbaa63c4ba None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:50.053553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7a707-dca2-40f3-9663-c6c483bd748b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:50.063079 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cadbc-8c7e-4058-ae3d-2e7475fc08d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:50.063752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cadbc-8c7e-4058-ae3d-2e7475fc08d3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:50.064042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cadbc-8c7e-4058-ae3d-2e7475fc08d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:50.064324 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cadbc-8c7e-4058-ae3d-2e7475fc08d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:50.065520 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cadbc-8c7e-4058-ae3d-2e7475fc08d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:50.065820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cadbc-8c7e-4058-ae3d-2e7475fc08d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:50.067836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cadbc-8c7e-4058-ae3d-2e7475fc08d3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:50.084507 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 301/598] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:21:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:50.084507 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cadbc-8c7e-4058-ae3d-2e7475fc08d3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:50.087142 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:50.094820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97d841-45eb-4451-8278-000e2e7a78e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:50.095692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97d841-45eb-4451-8278-000e2e7a78e0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:50.096042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97d841-45eb-4451-8278-000e2e7a78e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:50.096382 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97d841-45eb-4451-8278-000e2e7a78e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:50.107549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97d841-45eb-4451-8278-000e2e7a78e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:50.109113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97d841-45eb-4451-8278-000e2e7a78e0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:50.126753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-ff97d841-45eb-4451-8278-000e2e7a78e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:50.126753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:50.126753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:50.126753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:50.126753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:50.126753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:50.126753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:50.126753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:50.126753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:50.126753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:50.126753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:50.126753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:50.126753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:50.126753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:50.126753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:50.127697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:50.129274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:50.129274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:50.129274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:50.129274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:50.129274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 298/599] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:50.130036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb9fc2-761b-46bb-9900-2c83f567e418 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:50.130527 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb9fc2-761b-46bb-9900-2c83f567e418 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:50.130527 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb9fc2-761b-46bb-9900-2c83f567e418 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:50.131933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 299/600] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:50.134038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb9fc2-761b-46bb-9900-2c83f567e418 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:50.147620 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:50.193932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cadbc-8c7e-4058-ae3d-2e7475fc08d3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:50.229983 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cefa0-5e03-4968-82e3-e1a89e1ca16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:50.230834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cefa0-5e03-4968-82e3-e1a89e1ca16a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:50.231098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cefa0-5e03-4968-82e3-e1a89e1ca16a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:50.231371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cefa0-5e03-4968-82e3-e1a89e1ca16a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:50.238078 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cefa0-5e03-4968-82e3-e1a89e1ca16a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:50.239642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cefa0-5e03-4968-82e3-e1a89e1ca16a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:50.242374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb9fc2-761b-46bb-9900-2c83f567e418 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:50.257107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-1c0cefa0-5e03-4968-82e3-e1a89e1ca16a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:50.257107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:50.257107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:50.257107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:50.257107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:50.257107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:50.257107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:50.257107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:50.257107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:50.257107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:50.257107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:50.257107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:50.257107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:21:50.257107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:21:50.257107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:50.258064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:50.258960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:21:50.258960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:21:50.258960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:21:50.258960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:21:50.258960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:21:50.258960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:21:50.258960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:50.258960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:50.258960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 302/601] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:50.258960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cefa0-5e03-4968-82e3-e1a89e1ca16a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:50.284456 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432ac8d-fa0c-41ef-9c17-66f98f548b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:50.286299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432ac8d-fa0c-41ef-9c17-66f98f548b34 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:50.286601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432ac8d-fa0c-41ef-9c17-66f98f548b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:50.286867 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432ac8d-fa0c-41ef-9c17-66f98f548b34 None admin] PATH_INFO: `/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/45ee7a6c342548f98e7dc73f4f71a66d/roles/3d6e986dd2264e3cb7e01e25dbac6f97` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:50.303448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432ac8d-fa0c-41ef-9c17-66f98f548b34 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d81eafac474f37a71ee5cc292d172f, user_id=45ee7a6c342548f98e7dc73f4f71a66d, role_id=3d6e986dd2264e3cb7e01e25dbac6f97)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:50.304900 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432ac8d-fa0c-41ef-9c17-66f98f548b34 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:50.322431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:50.359703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cefa0-5e03-4968-82e3-e1a89e1ca16a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:50.392489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2031e-f2ca-4f3b-a349-e6be464bcbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:50.393255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2031e-f2ca-4f3b-a349-e6be464bcbf2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:50.393543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2031e-f2ca-4f3b-a349-e6be464bcbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:50.393822 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2031e-f2ca-4f3b-a349-e6be464bcbf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:50.395283 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2031e-f2ca-4f3b-a349-e6be464bcbf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:50.395599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2031e-f2ca-4f3b-a349-e6be464bcbf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:50.396993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2031e-f2ca-4f3b-a349-e6be464bcbf2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:50.401372 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 300/602] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:21:50 2022] PUT /identity/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/45ee7a6c342548f98e7dc73f4f71a66d/roles/3d6e986dd2264e3cb7e01e25dbac6f97 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:21:50.403010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb7324-e4e8-41f7-a1ed-6579b97ce5cd None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:50.403371 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb7324-e4e8-41f7-a1ed-6579b97ce5cd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:50.403660 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb7324-e4e8-41f7-a1ed-6579b97ce5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:50.410802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 303/603] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:50.411355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2031e-f2ca-4f3b-a349-e6be464bcbf2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:50.445043 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-75eb7324-e4e8-41f7-a1ed-6579b97ce5cd None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:50.445897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-75eb7324-e4e8-41f7-a1ed-6579b97ce5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:50.510746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:50.529361 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 301/604] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:50.530733 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb7324-e4e8-41f7-a1ed-6579b97ce5cd None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:50.542896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2031e-f2ca-4f3b-a349-e6be464bcbf2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:50.574725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:50.597554 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b9c54-13b2-4723-a5e7-c6b6d35d5df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:50.598413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b9c54-13b2-4723-a5e7-c6b6d35d5df6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:50.598704 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b9c54-13b2-4723-a5e7-c6b6d35d5df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:50.598989 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b9c54-13b2-4723-a5e7-c6b6d35d5df6 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:50.600216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b9c54-13b2-4723-a5e7-c6b6d35d5df6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:50.600543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b9c54-13b2-4723-a5e7-c6b6d35d5df6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:50.602679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b9c54-13b2-4723-a5e7-c6b6d35d5df6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:50.615945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 304/605] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:21:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:50.616526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b9c54-13b2-4723-a5e7-c6b6d35d5df6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:50.665912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb7324-e4e8-41f7-a1ed-6579b97ce5cd None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:50.703649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad9dd1-c1f3-4443-ae7f-a91dff9802d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:50.704815 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad9dd1-c1f3-4443-ae7f-a91dff9802d5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:50.705231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad9dd1-c1f3-4443-ae7f-a91dff9802d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:50.705608 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad9dd1-c1f3-4443-ae7f-a91dff9802d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:50.714267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad9dd1-c1f3-4443-ae7f-a91dff9802d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:50.715859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad9dd1-c1f3-4443-ae7f-a91dff9802d5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:50.731308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-daad9dd1-c1f3-4443-ae7f-a91dff9802d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:50.731308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:50.731308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:50.731308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:50.731308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:50.731308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:50.731308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:50.731308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:50.731308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:50.731308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:50.731308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:50.731308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:50.731308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:21:50.731308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:21:50.731308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:50.732417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:50.733549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:21:50.733549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:21:50.733549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:21:50.733549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:21:50.733549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:21:50.733549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:21:50.733549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:50.733549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:50.735320 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 302/606] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:50.736261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad9dd1-c1f3-4443-ae7f-a91dff9802d5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:50.758698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b9c54-13b2-4723-a5e7-c6b6d35d5df6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:50.802525 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3236d8-3f93-414c-ab40-cbc3d40ca6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:50.804269 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3236d8-3f93-414c-ab40-cbc3d40ca6ac None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:50.804728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3236d8-3f93-414c-ab40-cbc3d40ca6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:50.805430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3236d8-3f93-414c-ab40-cbc3d40ca6ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:50.808069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:50.821107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3236d8-3f93-414c-ab40-cbc3d40ca6ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-4e3236d8-3f93-414c-ab40-cbc3d40ca6ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:50.832312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:50.833478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:50.834517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:50.834517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:50.834517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:50.834517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:50.834517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:50.834517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:50.834517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:50.834517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:50.834517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:50.834517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:50.834517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:50.834517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:50.841866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 305/607] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:50.844341 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3236d8-3f93-414c-ab40-cbc3d40ca6ac None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:50.880618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad9dd1-c1f3-4443-ae7f-a91dff9802d5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:50.890956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:50.930545 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99991b-6a6e-4424-9b1d-4d81b40a5055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:50.932147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99991b-6a6e-4424-9b1d-4d81b40a5055 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:50.932585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99991b-6a6e-4424-9b1d-4d81b40a5055 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:50.933299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99991b-6a6e-4424-9b1d-4d81b40a5055 None admin] PATH_INFO: `/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/8750b0450bf343f590c52b7d5c68d5df/roles/3d6e986dd2264e3cb7e01e25dbac6f97` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:50.953897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99991b-6a6e-4424-9b1d-4d81b40a5055 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d81eafac474f37a71ee5cc292d172f, user_id=8750b0450bf343f590c52b7d5c68d5df, role_id=3d6e986dd2264e3cb7e01e25dbac6f97)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:50.955596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99991b-6a6e-4424-9b1d-4d81b40a5055 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:50.955815 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3236d8-3f93-414c-ab40-cbc3d40ca6ac None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:50.994254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286949f-ed62-44f4-920d-9ccde15fcc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:50.995087 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286949f-ed62-44f4-920d-9ccde15fcc4d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:50.995472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286949f-ed62-44f4-920d-9ccde15fcc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:50.995847 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286949f-ed62-44f4-920d-9ccde15fcc4d None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:50.997243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286949f-ed62-44f4-920d-9ccde15fcc4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:50.997650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286949f-ed62-44f4-920d-9ccde15fcc4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:50.999156 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286949f-ed62-44f4-920d-9ccde15fcc4d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:51.012105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 306/608] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:21:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:51.012931 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286949f-ed62-44f4-920d-9ccde15fcc4d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:51.037887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 303/609] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:21:50 2022] PUT /identity/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/8750b0450bf343f590c52b7d5c68d5df/roles/3d6e986dd2264e3cb7e01e25dbac6f97 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:21:51.039412 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99991b-6a6e-4424-9b1d-4d81b40a5055 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:51.170874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:51.175592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99991b-6a6e-4424-9b1d-4d81b40a5055 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:51.220089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccafe1-1773-4d50-b5bd-38590639a927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:51.221375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccafe1-1773-4d50-b5bd-38590639a927 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:51.222944 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccafe1-1773-4d50-b5bd-38590639a927 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:51.223418 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccafe1-1773-4d50-b5bd-38590639a927 None admin] PATH_INFO: `/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/12ceb8a355a44b9fb29918483379cb38/roles/3d6e986dd2264e3cb7e01e25dbac6f97` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:51.231396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286949f-ed62-44f4-920d-9ccde15fcc4d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:51.233437 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:51.242793 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccafe1-1773-4d50-b5bd-38590639a927 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d81eafac474f37a71ee5cc292d172f, user_id=12ceb8a355a44b9fb29918483379cb38, role_id=3d6e986dd2264e3cb7e01e25dbac6f97)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:51.245236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccafe1-1773-4d50-b5bd-38590639a927 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:51.287405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c5e2e-cb10-482f-a006-3e7a201a2967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:51.288491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c5e2e-cb10-482f-a006-3e7a201a2967 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:51.288979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c5e2e-cb10-482f-a006-3e7a201a2967 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:51.289508 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c5e2e-cb10-482f-a006-3e7a201a2967 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:51.290526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c5e2e-cb10-482f-a006-3e7a201a2967 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:51.291109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c5e2e-cb10-482f-a006-3e7a201a2967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:51.292856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c5e2e-cb10-482f-a006-3e7a201a2967 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:51.309043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 307/610] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:21:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:51.311025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c5e2e-cb10-482f-a006-3e7a201a2967 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:51.336089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 304/611] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:21:51 2022] PUT /identity/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/12ceb8a355a44b9fb29918483379cb38/roles/3d6e986dd2264e3cb7e01e25dbac6f97 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:21:51.337758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f0b3e-279d-40c1-92f4-a40dfa6cc329 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:51.338910 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f0b3e-279d-40c1-92f4-a40dfa6cc329 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:51.339435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f0b3e-279d-40c1-92f4-a40dfa6cc329 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:51.341898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 305/612] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:51.351460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f32d0-e006-40cd-95bf-7e973f911782 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:51.352133 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f32d0-e006-40cd-95bf-7e973f911782 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:51.352800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f32d0-e006-40cd-95bf-7e973f911782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:51.382083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7a5f32d0-e006-40cd-95bf-7e973f911782 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:51.383901 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7a5f32d0-e006-40cd-95bf-7e973f911782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:51.449318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c5e2e-cb10-482f-a006-3e7a201a2967 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:51.491141 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 306/613] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:21:51.514991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f3954-1ff5-498b-8fe3-fa170f290201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:51.515811 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f3954-1ff5-498b-8fe3-fa170f290201 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:51.516098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f3954-1ff5-498b-8fe3-fa170f290201 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:51.516376 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f3954-1ff5-498b-8fe3-fa170f290201 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:51.529253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f32d0-e006-40cd-95bf-7e973f911782 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:51.553712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f3954-1ff5-498b-8fe3-fa170f290201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:51.555489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f3954-1ff5-498b-8fe3-fa170f290201 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:51.583222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-885f3954-1ff5-498b-8fe3-fa170f290201 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:51.583222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:51.583222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:51.583222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:51.583222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:51.583222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:51.583222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:51.583222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:51.583222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:51.583222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:51.583222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:51.583222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:51.583222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:51.583222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:51.583222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:51.584193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:51.585164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:51.585164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:51.585164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:51.585164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:51.585164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 308/614] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:51.590554 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f3954-1ff5-498b-8fe3-fa170f290201 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:51.619508 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:51.645334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:51.665882 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f32d0-e006-40cd-95bf-7e973f911782 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:51.705160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665c555-3ee5-430d-8508-d9c53ee918f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:51.706067 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665c555-3ee5-430d-8508-d9c53ee918f5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:51.706180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665c555-3ee5-430d-8508-d9c53ee918f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:51.706578 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665c555-3ee5-430d-8508-d9c53ee918f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:51.716840 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f3954-1ff5-498b-8fe3-fa170f290201 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:51.717776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665c555-3ee5-430d-8508-d9c53ee918f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-2665c555-3ee5-430d-8508-d9c53ee918f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:51.725923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:51.726951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:51.727827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:51.727827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:51.727827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:51.727827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:51.727827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:51.727827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:51.727827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:51.727827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:51.727827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:51.727827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:51.727827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:51.727827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:51.727827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 307/615] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:51.733369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665c555-3ee5-430d-8508-d9c53ee918f5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:51.754792 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa6120-c2ef-4733-95f2-8909244207bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:51.755527 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa6120-c2ef-4733-95f2-8909244207bb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:51.755814 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa6120-c2ef-4733-95f2-8909244207bb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:51.756093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa6120-c2ef-4733-95f2-8909244207bb None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:51.756965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa6120-c2ef-4733-95f2-8909244207bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:51.757347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa6120-c2ef-4733-95f2-8909244207bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:51.758782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa6120-c2ef-4733-95f2-8909244207bb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:51.772772 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 309/616] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:51.777733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa6120-c2ef-4733-95f2-8909244207bb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:51.837948 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665c555-3ee5-430d-8508-d9c53ee918f5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:51.863405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:51.875426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a82ca-4e2c-4139-b0fe-2ddbf653bb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:51.876110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a82ca-4e2c-4139-b0fe-2ddbf653bb19 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:51.876548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a82ca-4e2c-4139-b0fe-2ddbf653bb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:51.876805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a82ca-4e2c-4139-b0fe-2ddbf653bb19 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:51.877760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a82ca-4e2c-4139-b0fe-2ddbf653bb19 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:51.877937 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a82ca-4e2c-4139-b0fe-2ddbf653bb19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:51.879906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a82ca-4e2c-4139-b0fe-2ddbf653bb19 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:51.888826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 308/617] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:21:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:51.891143 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa6120-c2ef-4733-95f2-8909244207bb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:51.894853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a82ca-4e2c-4139-b0fe-2ddbf653bb19 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:51.897586 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:51.930086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3208103-fa7e-4dde-b54b-3c84480f7d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:51.931404 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3208103-fa7e-4dde-b54b-3c84480f7d7b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:51.931871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3208103-fa7e-4dde-b54b-3c84480f7d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:51.932333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3208103-fa7e-4dde-b54b-3c84480f7d7b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:51.940028 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3208103-fa7e-4dde-b54b-3c84480f7d7b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:51.942372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3208103-fa7e-4dde-b54b-3c84480f7d7b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-e3208103-fa7e-4dde-b54b-3c84480f7d7b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:51.958300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:51.959301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:51.960260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:51.960260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:51.960260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:51.960260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:51.960260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:51.960260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:51.960260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:51.960260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:51.960260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:21:51.960260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:51.962647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 310/618] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:21:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:21:51.974120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3208103-fa7e-4dde-b54b-3c84480f7d7b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:52.013103 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a82ca-4e2c-4139-b0fe-2ddbf653bb19 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:52.054059 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed17880-b887-491b-97d3-e9ee1e807849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:52.054876 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed17880-b887-491b-97d3-e9ee1e807849 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:52.056086 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed17880-b887-491b-97d3-e9ee1e807849 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:52.056086 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed17880-b887-491b-97d3-e9ee1e807849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:52.071152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed17880-b887-491b-97d3-e9ee1e807849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:52.073797 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed17880-b887-491b-97d3-e9ee1e807849 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:52.099466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-0ed17880-b887-491b-97d3-e9ee1e807849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:52.099466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:52.099466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:52.099466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:52.099466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:52.099466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:52.099466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:52.099466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:52.099466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:52.099466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:52.099466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:52.099466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:52.099466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:52.099466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:52.099466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:52.100445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:52.101472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:52.101472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:52.101472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:52.101472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:52.101472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 309/619] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:52.114766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed17880-b887-491b-97d3-e9ee1e807849 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:52.180790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3208103-fa7e-4dde-b54b-3c84480f7d7b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:52.232208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:52.257736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bbaf0-1776-4b30-a797-8a9822781c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:52.258467 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bbaf0-1776-4b30-a797-8a9822781c5d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:52.258752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bbaf0-1776-4b30-a797-8a9822781c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:52.259026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bbaf0-1776-4b30-a797-8a9822781c5d None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:52.260979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4bbaf0-1776-4b30-a797-8a9822781c5d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:52.261317 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4bbaf0-1776-4b30-a797-8a9822781c5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:52.262748 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4bbaf0-1776-4b30-a797-8a9822781c5d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:52.273380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:52.307089 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 311/620] 10.222.0.64 () {56 vars in 1135 bytes} [Mon May 23 15:21:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:52.308970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bbaf0-1776-4b30-a797-8a9822781c5d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:52.335988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed17880-b887-491b-97d3-e9ee1e807849 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:52.380942 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a93a1-87c8-4a07-8d5c-bfcb134746bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:52.381652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a93a1-87c8-4a07-8d5c-bfcb134746bc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:52.381939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a93a1-87c8-4a07-8d5c-bfcb134746bc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:52.382218 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a93a1-87c8-4a07-8d5c-bfcb134746bc None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:52.383032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a93a1-87c8-4a07-8d5c-bfcb134746bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:52.383352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a93a1-87c8-4a07-8d5c-bfcb134746bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:52.384731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a93a1-87c8-4a07-8d5c-bfcb134746bc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:52.411365 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 310/621] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:52.417145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a93a1-87c8-4a07-8d5c-bfcb134746bc None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:52.434492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bbaf0-1776-4b30-a797-8a9822781c5d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:52.489868 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b9b45-3043-402f-a43d-2a14f4dc0b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:52.490614 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b9b45-3043-402f-a43d-2a14f4dc0b8d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:52.490907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b9b45-3043-402f-a43d-2a14f4dc0b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:52.491188 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b9b45-3043-402f-a43d-2a14f4dc0b8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:52.506559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b9b45-3043-402f-a43d-2a14f4dc0b8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:52.508027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b9b45-3043-402f-a43d-2a14f4dc0b8d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:52.542978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a93a1-87c8-4a07-8d5c-bfcb134746bc None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:52.544534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-2c1b9b45-3043-402f-a43d-2a14f4dc0b8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:52.544534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:52.544534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:52.544534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:52.544534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:52.544534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:52.544534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:52.544534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:52.544534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:52.544534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:52.544534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:52.544534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:52.544534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:52.544534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:52.544534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:52.546509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:52.548590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:52.548590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:52.548590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:52.548590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:52.553952 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 312/622] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:52.558580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:52.559302 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b9b45-3043-402f-a43d-2a14f4dc0b8d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:52.579026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4fac2-abe5-43e5-8935-8b43414419cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:52.579859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4fac2-abe5-43e5-8935-8b43414419cc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:52.580160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4fac2-abe5-43e5-8935-8b43414419cc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:52.580448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4fac2-abe5-43e5-8935-8b43414419cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:52.588131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4fac2-abe5-43e5-8935-8b43414419cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:52.589630 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4fac2-abe5-43e5-8935-8b43414419cc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-12e4fac2-abe5-43e5-8935-8b43414419cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:52.608675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:52.610990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:52.613727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:52.613727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:52.613727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:52.613727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:52.613727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:52.613727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:52.613727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:52.613727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:52.613727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:21:52.613727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:52.613727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 311/623] 10.222.0.64 () {56 vars in 1093 bytes} [Mon May 23 15:21:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:21:52.617382 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4fac2-abe5-43e5-8935-8b43414419cc None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:52.667117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:52.758785 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4fac2-abe5-43e5-8935-8b43414419cc None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:52.789520 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b9b45-3043-402f-a43d-2a14f4dc0b8d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:52.835550 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3122e-559d-4fbd-8b38-92ad39ea0eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:52.836314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3122e-559d-4fbd-8b38-92ad39ea0eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:52.836608 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3122e-559d-4fbd-8b38-92ad39ea0eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:52.836890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3122e-559d-4fbd-8b38-92ad39ea0eb2 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:52.837895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3122e-559d-4fbd-8b38-92ad39ea0eb2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:52.838211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3122e-559d-4fbd-8b38-92ad39ea0eb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:52.839694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3122e-559d-4fbd-8b38-92ad39ea0eb2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:52.864324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 312/624] 10.222.0.64 () {56 vars in 1133 bytes} [Mon May 23 15:21:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:52.869331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3122e-559d-4fbd-8b38-92ad39ea0eb2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:52.871666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e08b4-f68a-4463-a858-22e3a47a9bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:52.872359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e08b4-f68a-4463-a858-22e3a47a9bfc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:52.872652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e08b4-f68a-4463-a858-22e3a47a9bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:52.872937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e08b4-f68a-4463-a858-22e3a47a9bfc None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:52.873715 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e08b4-f68a-4463-a858-22e3a47a9bfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:52.874114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e08b4-f68a-4463-a858-22e3a47a9bfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:52.875491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e08b4-f68a-4463-a858-22e3a47a9bfc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:52.892501 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 313/625] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:52.894007 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc33387-2878-492a-9cbd-4f9dcf619b69 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:52.894544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc33387-2878-492a-9cbd-4f9dcf619b69 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:52.894843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc33387-2878-492a-9cbd-4f9dcf619b69 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:52.896411 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 314/626] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:52.901316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc33387-2878-492a-9cbd-4f9dcf619b69 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:52.910000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:52.996792 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3122e-559d-4fbd-8b38-92ad39ea0eb2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:53.032521 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc33387-2878-492a-9cbd-4f9dcf619b69 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:53.033984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358f306-934a-4f06-80c1-cb64c1c09e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:53.034974 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358f306-934a-4f06-80c1-cb64c1c09e46 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:53.035491 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358f306-934a-4f06-80c1-cb64c1c09e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:53.035924 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358f306-934a-4f06-80c1-cb64c1c09e46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:53.041040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:53.081817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f679224-f4a6-405f-b898-b2e3c2c61e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:53.082615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f679224-f4a6-405f-b898-b2e3c2c61e6c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:53.082907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f679224-f4a6-405f-b898-b2e3c2c61e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:53.083187 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f679224-f4a6-405f-b898-b2e3c2c61e6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:53.091907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f679224-f4a6-405f-b898-b2e3c2c61e6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:53.094234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f679224-f4a6-405f-b898-b2e3c2c61e6c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:53.114208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-4f679224-f4a6-405f-b898-b2e3c2c61e6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:53.114208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:53.114208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:53.114208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:53.114208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:53.114208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:53.114208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:53.114208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:53.114208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:53.114208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:53.114208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:53.114208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:53.114208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:21:53.114208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:21:53.114208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:53.115198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:53.116120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:21:53.116120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:21:53.116120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:21:53.116120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:21:53.116120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:21:53.116120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:21:53.116120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:53.116120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:53.119841 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 315/627] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:53.120225 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358f306-934a-4f06-80c1-cb64c1c09e46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:53.121002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0dab1-73f2-4fc5-93f0-c9d4ab6cb810 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:53.121322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0dab1-73f2-4fc5-93f0-c9d4ab6cb810 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:53.121599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0dab1-73f2-4fc5-93f0-c9d4ab6cb810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:53.121757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358f306-934a-4f06-80c1-cb64c1c09e46 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:53.146831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e2b0dab1-73f2-4fc5-93f0-c9d4ab6cb810 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:53.148516 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e2b0dab1-73f2-4fc5-93f0-c9d4ab6cb810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:53.160007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:53.224528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 316/628] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:53.225750 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0dab1-73f2-4fc5-93f0-c9d4ab6cb810 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:53.256485 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 313/629] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:21:53.257785 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358f306-934a-4f06-80c1-cb64c1c09e46 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:53.347714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:53.357947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0dab1-73f2-4fc5-93f0-c9d4ab6cb810 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:53.383928 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358f306-934a-4f06-80c1-cb64c1c09e46 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:53.410722 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7608aa-9f3e-4f81-b1c9-8636f28745aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:53.411402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7608aa-9f3e-4f81-b1c9-8636f28745aa None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:53.411695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7608aa-9f3e-4f81-b1c9-8636f28745aa None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:53.411976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7608aa-9f3e-4f81-b1c9-8636f28745aa None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:53.412778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7608aa-9f3e-4f81-b1c9-8636f28745aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:53.413177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7608aa-9f3e-4f81-b1c9-8636f28745aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:53.414641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7608aa-9f3e-4f81-b1c9-8636f28745aa None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:53.426727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:53.433207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c03799-4901-4c61-af41-8e3213561118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:53.434260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 317/630] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:21:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:53.434515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c03799-4901-4c61-af41-8e3213561118 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:53.435069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c03799-4901-4c61-af41-8e3213561118 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:53.435191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7608aa-9f3e-4f81-b1c9-8636f28745aa None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:53.435756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c03799-4901-4c61-af41-8e3213561118 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:53.451833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c03799-4901-4c61-af41-8e3213561118 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-75c03799-4901-4c61-af41-8e3213561118 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:53.464875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:53.465915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:53.466763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:53.466763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:53.466763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:53.466763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:53.466763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:53.466763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:53.466763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:53.466763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:53.466763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:53.466763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:53.466763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:53.466763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:53.469776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 314/631] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:53.470746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c03799-4901-4c61-af41-8e3213561118 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:53.580423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c03799-4901-4c61-af41-8e3213561118 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:53.582313 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7608aa-9f3e-4f81-b1c9-8636f28745aa None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:53.621505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94bd02-6992-4679-8243-a8ab75347c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:53.621505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94bd02-6992-4679-8243-a8ab75347c30 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:53.621505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94bd02-6992-4679-8243-a8ab75347c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:53.621505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94bd02-6992-4679-8243-a8ab75347c30 None admin] PATH_INFO: `/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/f8c79c36c7204368816593f0e473191b/roles/3d6e986dd2264e3cb7e01e25dbac6f97` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:53.631348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7bee9-7e18-44fe-920d-c28e708feba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:53.632499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7bee9-7e18-44fe-920d-c28e708feba8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:53.633216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7bee9-7e18-44fe-920d-c28e708feba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:53.633642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7bee9-7e18-44fe-920d-c28e708feba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:53.637217 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94bd02-6992-4679-8243-a8ab75347c30 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d81eafac474f37a71ee5cc292d172f, user_id=f8c79c36c7204368816593f0e473191b, role_id=3d6e986dd2264e3cb7e01e25dbac6f97)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:53.638846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94bd02-6992-4679-8243-a8ab75347c30 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:53.647967 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7bee9-7e18-44fe-920d-c28e708feba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:53.649772 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7bee9-7e18-44fe-920d-c28e708feba8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:53.667768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-69f7bee9-7e18-44fe-920d-c28e708feba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:53.667768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:53.667768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:53.667768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:53.667768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:53.667768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:53.667768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:53.667768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:53.667768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:53.667768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:53.667768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:53.667768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:53.667768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:53.667768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:53.667768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:53.668820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:53.673718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:53.673718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:53.673718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:53.673718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:53.673718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 318/632] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:53.673718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7bee9-7e18-44fe-920d-c28e708feba8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:53.684514 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:53.719560 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 315/633] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:21:53 2022] PUT /identity/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/f8c79c36c7204368816593f0e473191b/roles/3d6e986dd2264e3cb7e01e25dbac6f97 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:21:53.721092 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94bd02-6992-4679-8243-a8ab75347c30 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:53.830546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:53.877283 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7bee9-7e18-44fe-920d-c28e708feba8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:53.928544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57657b5a-b4ff-4484-995d-8a5ad65fc9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:53.929288 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57657b5a-b4ff-4484-995d-8a5ad65fc9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:53.929577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57657b5a-b4ff-4484-995d-8a5ad65fc9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:53.929860 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57657b5a-b4ff-4484-995d-8a5ad65fc9d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:53.930629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57657b5a-b4ff-4484-995d-8a5ad65fc9d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:53.931037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57657b5a-b4ff-4484-995d-8a5ad65fc9d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:53.932553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57657b5a-b4ff-4484-995d-8a5ad65fc9d2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:53.943216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 319/634] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:21:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:53.944701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fa837-570d-4c9b-9f8a-64829d0ffea2 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:53.945000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fa837-570d-4c9b-9f8a-64829d0ffea2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:53.945308 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fa837-570d-4c9b-9f8a-64829d0ffea2 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:53.946929 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 320/635] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:53.951993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605bea8-7a33-40c7-9b05-1b13ff8e83bc None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:53.952290 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605bea8-7a33-40c7-9b05-1b13ff8e83bc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:53.952580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605bea8-7a33-40c7-9b05-1b13ff8e83bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:53.955709 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94bd02-6992-4679-8243-a8ab75347c30 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:53.971544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b605bea8-7a33-40c7-9b05-1b13ff8e83bc None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:53.973529 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b605bea8-7a33-40c7-9b05-1b13ff8e83bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:54.000555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdb8d9-506e-497c-8741-08146b7224a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:54.001366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdb8d9-506e-497c-8741-08146b7224a2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:54.001570 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdb8d9-506e-497c-8741-08146b7224a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:54.001855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdb8d9-506e-497c-8741-08146b7224a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:54.002748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecdb8d9-506e-497c-8741-08146b7224a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:54.002940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecdb8d9-506e-497c-8741-08146b7224a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:54.005030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecdb8d9-506e-497c-8741-08146b7224a2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:54.016775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 316/636] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:54.017464 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdb8d9-506e-497c-8741-08146b7224a2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:54.055409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:54.065096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 321/637] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:54.065096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92feabdd-9920-471e-a10c-d6570365670f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:54.065096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92feabdd-9920-471e-a10c-d6570365670f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:54.065096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92feabdd-9920-471e-a10c-d6570365670f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:54.074396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 322/638] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:54.074396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92feabdd-9920-471e-a10c-d6570365670f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:54.139004 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdb8d9-506e-497c-8741-08146b7224a2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:54.140576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:54.176641 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93cc0d-8178-43d1-beea-5f1f3681d56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:54.177725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92feabdd-9920-471e-a10c-d6570365670f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:54.177838 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93cc0d-8178-43d1-beea-5f1f3681d56d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:54.178029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93cc0d-8178-43d1-beea-5f1f3681d56d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:54.178297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93cc0d-8178-43d1-beea-5f1f3681d56d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:54.190103 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93cc0d-8178-43d1-beea-5f1f3681d56d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:54.192769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93cc0d-8178-43d1-beea-5f1f3681d56d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:54.217997 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae59c80-a98d-4509-b439-2f8ef3fa73b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:54.218926 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae59c80-a98d-4509-b439-2f8ef3fa73b8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:54.219210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae59c80-a98d-4509-b439-2f8ef3fa73b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:54.219489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae59c80-a98d-4509-b439-2f8ef3fa73b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:54.227564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae59c80-a98d-4509-b439-2f8ef3fa73b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:54.229041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae59c80-a98d-4509-b439-2f8ef3fa73b8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:54.244259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6ae59c80-a98d-4509-b439-2f8ef3fa73b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:54.244259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:54.244259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:54.244259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:54.244259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:54.244259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:54.244259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:54.244259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:54.244259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:54.244259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:54.244259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:54.244259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:54.244259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:21:54.244259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:21:54.244259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:54.245640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:54.246611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:21:54.246611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:21:54.246611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:21:54.246611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:21:54.246611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:21:54.246611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:21:54.246611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:54.246611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:54.246611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 323/639] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:54.247193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3988c0d-03a8-4beb-9efd-b45b7fcf4835 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:54.247617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3988c0d-03a8-4beb-9efd-b45b7fcf4835 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:54.247715 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3988c0d-03a8-4beb-9efd-b45b7fcf4835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:54.266316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c3988c0d-03a8-4beb-9efd-b45b7fcf4835 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:54.267530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c3988c0d-03a8-4beb-9efd-b45b7fcf4835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:54.331597 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:54.335535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-ff93cc0d-8178-43d1-beea-5f1f3681d56d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:54.335535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:54.335535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:54.335535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:54.335535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:54.335535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:54.335535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:54.335535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:54.335535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:54.335535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:54.335535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:54.335535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:54.335535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:54.335535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:54.335535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:54.336476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:54.338189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:54.338189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:54.338189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:54.338189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:54.338189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 317/640] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:54.338189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93cc0d-8178-43d1-beea-5f1f3681d56d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:54.355747 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 324/641] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:54.355747 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3988c0d-03a8-4beb-9efd-b45b7fcf4835 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:54.463467 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3988c0d-03a8-4beb-9efd-b45b7fcf4835 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:54.463647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93cc0d-8178-43d1-beea-5f1f3681d56d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:54.497424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad37d5-5232-4098-af3c-a8ca0bc45406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:54.498227 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad37d5-5232-4098-af3c-a8ca0bc45406 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:54.498516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad37d5-5232-4098-af3c-a8ca0bc45406 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:54.498795 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad37d5-5232-4098-af3c-a8ca0bc45406 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:54.505797 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959c3d2-c4f4-4729-8c22-1cf1508275f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:54.506708 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959c3d2-c4f4-4729-8c22-1cf1508275f5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:54.507148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959c3d2-c4f4-4729-8c22-1cf1508275f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:54.507579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959c3d2-c4f4-4729-8c22-1cf1508275f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:54.508544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959c3d2-c4f4-4729-8c22-1cf1508275f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:54.509010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959c3d2-c4f4-4729-8c22-1cf1508275f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:54.509532 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad37d5-5232-4098-af3c-a8ca0bc45406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:54.510778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959c3d2-c4f4-4729-8c22-1cf1508275f5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-0fad37d5-5232-4098-af3c-a8ca0bc45406 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:54.517166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:54.518425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:54.519925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:54.519925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:54.519925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:54.519925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:54.519925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:54.519925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:54.519925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:54.519925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:54.519925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:54.519925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:54.519925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:54.519925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:54.519925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 318/642] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:54.520675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad37d5-5232-4098-af3c-a8ca0bc45406 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:54.529506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 325/643] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:21:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:54.530563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959c3d2-c4f4-4729-8c22-1cf1508275f5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:54.542395 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:54.587830 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:54.631759 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad37d5-5232-4098-af3c-a8ca0bc45406 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:54.640557 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959c3d2-c4f4-4729-8c22-1cf1508275f5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:54.667400 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f3554-a448-4a6a-a737-55a3048a4d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:54.668070 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f3554-a448-4a6a-a737-55a3048a4d62 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:54.668356 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f3554-a448-4a6a-a737-55a3048a4d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:54.668635 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f3554-a448-4a6a-a737-55a3048a4d62 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:54.669390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f3554-a448-4a6a-a737-55a3048a4d62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:54.669703 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f3554-a448-4a6a-a737-55a3048a4d62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:54.671074 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f3554-a448-4a6a-a737-55a3048a4d62 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:54.680582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcf1ee-c3bd-4124-bf26-ebf9057c8f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:54.681415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcf1ee-c3bd-4124-bf26-ebf9057c8f4d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:54.681772 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcf1ee-c3bd-4124-bf26-ebf9057c8f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:54.682047 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcf1ee-c3bd-4124-bf26-ebf9057c8f4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:54.683927 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 319/644] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:54.689086 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f3554-a448-4a6a-a737-55a3048a4d62 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:54.694485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfcf1ee-c3bd-4124-bf26-ebf9057c8f4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-8dfcf1ee-c3bd-4124-bf26-ebf9057c8f4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:54.704173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:54.705201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:54.711438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:54.711438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:54.711438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:54.711438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:54.711438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:54.711438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:54.711438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:54.711438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:54.711438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:54.711438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:54.711438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:54.711438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:54.711438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 326/645] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:54.711438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcf1ee-c3bd-4124-bf26-ebf9057c8f4d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:54.766210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:54.791496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f3554-a448-4a6a-a737-55a3048a4d62 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:54.820151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcf1ee-c3bd-4124-bf26-ebf9057c8f4d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:54.827694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16ac3d-6935-4dcc-b356-babd36b7791b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:54.828619 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16ac3d-6935-4dcc-b356-babd36b7791b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:54.829143 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16ac3d-6935-4dcc-b356-babd36b7791b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:54.829638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16ac3d-6935-4dcc-b356-babd36b7791b None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:54.830582 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16ac3d-6935-4dcc-b356-babd36b7791b None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:54.831318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:54.831542 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16ac3d-6935-4dcc-b356-babd36b7791b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:54.832933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16ac3d-6935-4dcc-b356-babd36b7791b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:54.846210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 320/646] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:21:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:54.846210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16ac3d-6935-4dcc-b356-babd36b7791b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:54.859210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e589fc-a569-4315-b088-7f5afb0594b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:54.860042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e589fc-a569-4315-b088-7f5afb0594b0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:54.860330 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e589fc-a569-4315-b088-7f5afb0594b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:54.881197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e589fc-a569-4315-b088-7f5afb0594b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:54.881197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e589fc-a569-4315-b088-7f5afb0594b0 None admin] RBAC: Adding query filter params (role.id=3d6e986dd2264e3cb7e01e25dbac6f97, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=8da96bed73794fee8f719ec904936164) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:54.881197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e589fc-a569-4315-b088-7f5afb0594b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:54.881197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e589fc-a569-4315-b088-7f5afb0594b0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:54.881197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 327/647] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:21:54 2022] GET /identity/v3/role_assignments?user.id=8da96bed73794fee8f719ec904936164&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=3d6e986dd2264e3cb7e01e25dbac6f97 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:54.881666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e589fc-a569-4315-b088-7f5afb0594b0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:54.989896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16ac3d-6935-4dcc-b356-babd36b7791b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:55.000532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e589fc-a569-4315-b088-7f5afb0594b0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:55.027351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dac6df-e560-4431-9d67-f323bf0982d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:55.028195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dac6df-e560-4431-9d67-f323bf0982d9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:55.028586 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dac6df-e560-4431-9d67-f323bf0982d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:55.028994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dac6df-e560-4431-9d67-f323bf0982d9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:55.032955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:55.039683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dac6df-e560-4431-9d67-f323bf0982d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:55.041527 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dac6df-e560-4431-9d67-f323bf0982d9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:55.048447 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bf8dc-655f-4bbc-9c91-6913c6bd3cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:55.049266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bf8dc-655f-4bbc-9c91-6913c6bd3cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:55.049702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bf8dc-655f-4bbc-9c91-6913c6bd3cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:55.050218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bf8dc-655f-4bbc-9c91-6913c6bd3cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:55.051136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bf8dc-655f-4bbc-9c91-6913c6bd3cb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:55.051609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bf8dc-655f-4bbc-9c91-6913c6bd3cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:55.053446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bf8dc-655f-4bbc-9c91-6913c6bd3cb9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-d5dac6df-e560-4431-9d67-f323bf0982d9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:55.058348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:55.059387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:55.060422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:55.060422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:55.060422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:55.060422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:55.060422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:55.060422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:55.060422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:55.060422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:55.060422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 15:21:55.060422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:55.065184 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 321/648] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:55.065184 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dac6df-e560-4431-9d67-f323bf0982d9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:55.072225 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 328/649] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:21:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:55.073375 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bf8dc-655f-4bbc-9c91-6913c6bd3cb9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:55.120563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:55.194536 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dac6df-e560-4431-9d67-f323bf0982d9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:55.207770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bf8dc-655f-4bbc-9c91-6913c6bd3cb9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:55.237795 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e2a41-c504-4be7-8f1e-762f9e1ad63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:55.238704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e2a41-c504-4be7-8f1e-762f9e1ad63a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:55.239005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e2a41-c504-4be7-8f1e-762f9e1ad63a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:55.239289 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e2a41-c504-4be7-8f1e-762f9e1ad63a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:55.243226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf3973-9c45-4693-a05d-8a099d9aa346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:55.243905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf3973-9c45-4693-a05d-8a099d9aa346 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:55.244196 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf3973-9c45-4693-a05d-8a099d9aa346 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:55.244477 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf3973-9c45-4693-a05d-8a099d9aa346 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:55.245380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf3973-9c45-4693-a05d-8a099d9aa346 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:55.245694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf3973-9c45-4693-a05d-8a099d9aa346 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:55.247131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf3973-9c45-4693-a05d-8a099d9aa346 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:55.250542 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e2a41-c504-4be7-8f1e-762f9e1ad63a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:55.252120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e2a41-c504-4be7-8f1e-762f9e1ad63a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:55.265785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 329/650] 10.222.0.64 () {56 vars in 1137 bytes} [Mon May 23 15:21:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:55.266694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf3973-9c45-4693-a05d-8a099d9aa346 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:55.270831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c08e2a41-c504-4be7-8f1e-762f9e1ad63a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:55.270831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:55.270831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:55.270831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:55.270831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:55.270831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:55.270831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:55.270831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:55.270831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:55.270831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:55.270831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:55.270831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:55.270831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:55.270831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:55.270831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:55.271717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:55.272637 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:55.272637 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:55.272637 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:55.272637 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:55.272637 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 322/651] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:55.277451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e2a41-c504-4be7-8f1e-762f9e1ad63a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:55.338186 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:55.377455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:55.381742 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf3973-9c45-4693-a05d-8a099d9aa346 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:55.399246 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e2a41-c504-4be7-8f1e-762f9e1ad63a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:55.430720 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7f7ed-c72a-4202-a13e-c7cd235ab37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:55.432486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7f7ed-c72a-4202-a13e-c7cd235ab37b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:55.433242 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7f7ed-c72a-4202-a13e-c7cd235ab37b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:55.433711 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7f7ed-c72a-4202-a13e-c7cd235ab37b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:55.437527 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26707b-1293-4866-aa2b-bd420a308ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:55.438309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26707b-1293-4866-aa2b-bd420a308ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:55.438671 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26707b-1293-4866-aa2b-bd420a308ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:55.439023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26707b-1293-4866-aa2b-bd420a308ba7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:55.446606 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7f7ed-c72a-4202-a13e-c7cd235ab37b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:55.448334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7f7ed-c72a-4202-a13e-c7cd235ab37b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:55.472959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-edc7f7ed-c72a-4202-a13e-c7cd235ab37b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:55.472959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:55.472959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:55.472959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:55.472959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:55.472959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:55.472959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:55.472959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:55.472959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:55.472959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:55.472959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:55.472959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:55.472959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:55.472959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:55.472959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:55.473962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:55.474960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:55.474960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:55.474960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:55.474960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:55.481526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 330/652] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:55.482332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7f7ed-c72a-4202-a13e-c7cd235ab37b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:55.530127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26707b-1293-4866-aa2b-bd420a308ba7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:55.531365 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26707b-1293-4866-aa2b-bd420a308ba7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:55.583697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:55.617496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7f7ed-c72a-4202-a13e-c7cd235ab37b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:55.646072 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 323/653] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:55.647058 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26707b-1293-4866-aa2b-bd420a308ba7 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:55.667155 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585246a-9138-4559-8fcb-8d2a33e8b02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:55.668184 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585246a-9138-4559-8fcb-8d2a33e8b02f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:55.668675 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585246a-9138-4559-8fcb-8d2a33e8b02f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:55.669201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585246a-9138-4559-8fcb-8d2a33e8b02f None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:55.670682 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585246a-9138-4559-8fcb-8d2a33e8b02f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:55.671591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585246a-9138-4559-8fcb-8d2a33e8b02f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:55.673758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585246a-9138-4559-8fcb-8d2a33e8b02f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:55.690155 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 331/654] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:55.692112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585246a-9138-4559-8fcb-8d2a33e8b02f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:55.725431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:55.822991 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26707b-1293-4866-aa2b-bd420a308ba7 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:55.884965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585246a-9138-4559-8fcb-8d2a33e8b02f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:55.896928 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4f462-9e06-434e-a4e0-88f3f0104554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:55.897641 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f462-9e06-434e-a4e0-88f3f0104554 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:55.897946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f462-9e06-434e-a4e0-88f3f0104554 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:55.898227 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f462-9e06-434e-a4e0-88f3f0104554 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:55.898961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4f462-9e06-434e-a4e0-88f3f0104554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:55.899273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4f462-9e06-434e-a4e0-88f3f0104554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:55.900661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4f462-9e06-434e-a4e0-88f3f0104554 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:55.924704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 324/655] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:55.924704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4f462-9e06-434e-a4e0-88f3f0104554 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:55.941778 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:55.965806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f6031-b01b-4cb9-aef5-92299bc26349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:55.966670 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f6031-b01b-4cb9-aef5-92299bc26349 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:55.966961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f6031-b01b-4cb9-aef5-92299bc26349 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:55.967242 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f6031-b01b-4cb9-aef5-92299bc26349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:55.990993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f6031-b01b-4cb9-aef5-92299bc26349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:55.992569 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f6031-b01b-4cb9-aef5-92299bc26349 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:56.027000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-336f6031-b01b-4cb9-aef5-92299bc26349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:56.027000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:56.027000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:56.027000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:56.027000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:56.027000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:56.027000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:56.027000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:56.027000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:56.027000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:56.027000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:56.027000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:56.027000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:56.027000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:56.027000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:56.027973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:56.028902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:56.028902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:56.028902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:56.028902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:56.035348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 332/656] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:56.036013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f6031-b01b-4cb9-aef5-92299bc26349 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:56.086739 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4f462-9e06-434e-a4e0-88f3f0104554 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:56.122647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:56.132776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509ca4c-7f85-4ad1-a02c-94f380214ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:56.134013 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509ca4c-7f85-4ad1-a02c-94f380214ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:56.134307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509ca4c-7f85-4ad1-a02c-94f380214ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:56.134587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509ca4c-7f85-4ad1-a02c-94f380214ce2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:56.144421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509ca4c-7f85-4ad1-a02c-94f380214ce2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:56.146084 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509ca4c-7f85-4ad1-a02c-94f380214ce2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-d509ca4c-7f85-4ad1-a02c-94f380214ce2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:56.160446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:56.161498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:56.162466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:56.162466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:56.162466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:56.162466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:56.162466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:56.162466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:56.162466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:56.162466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:56.162466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:21:56.162466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:56.166516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 325/657] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:56.167486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509ca4c-7f85-4ad1-a02c-94f380214ce2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:56.184114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f6031-b01b-4cb9-aef5-92299bc26349 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:56.226852 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cab93-dcf0-4e82-8221-ec9356e67d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:56.228197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cab93-dcf0-4e82-8221-ec9356e67d21 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:56.228490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cab93-dcf0-4e82-8221-ec9356e67d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:56.228921 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cab93-dcf0-4e82-8221-ec9356e67d21 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:56.237978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cab93-dcf0-4e82-8221-ec9356e67d21 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:56.239450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cab93-dcf0-4e82-8221-ec9356e67d21 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:56.242546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-666cab93-dcf0-4e82-8221-ec9356e67d21 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:56.254326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:56.265911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:56.266881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:56.266881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:56.266881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:56.266881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:56.266881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:56.266881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:56.266881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:56.266881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:56.266881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:21:56.266881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:56.266881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 333/658] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:56.266881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae04ae2-be85-4d58-9438-0d9e897e2fc1 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:56.266881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae04ae2-be85-4d58-9438-0d9e897e2fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:56.267626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae04ae2-be85-4d58-9438-0d9e897e2fc1 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:56.267626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 334/659] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:56.267626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae04ae2-be85-4d58-9438-0d9e897e2fc1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:56.282925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509ca4c-7f85-4ad1-a02c-94f380214ce2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:56.319521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fb153-da84-46ac-900f-6b9b5ad1def1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:56.320197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fb153-da84-46ac-900f-6b9b5ad1def1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:56.320489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fb153-da84-46ac-900f-6b9b5ad1def1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:56.320775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fb153-da84-46ac-900f-6b9b5ad1def1 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:56.321537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fb153-da84-46ac-900f-6b9b5ad1def1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:56.321851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fb153-da84-46ac-900f-6b9b5ad1def1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:56.323219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fb153-da84-46ac-900f-6b9b5ad1def1 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:56.335524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 326/660] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:56.336344 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fb153-da84-46ac-900f-6b9b5ad1def1 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:56.379067 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae04ae2-be85-4d58-9438-0d9e897e2fc1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:56.395869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:56.413391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9ba91-58a6-43f4-a9fd-b4abc37a44c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:56.414071 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9ba91-58a6-43f4-a9fd-b4abc37a44c8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:56.414363 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9ba91-58a6-43f4-a9fd-b4abc37a44c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:56.414649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9ba91-58a6-43f4-a9fd-b4abc37a44c8 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:56.415488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9ba91-58a6-43f4-a9fd-b4abc37a44c8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:56.415807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9ba91-58a6-43f4-a9fd-b4abc37a44c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:56.417291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9ba91-58a6-43f4-a9fd-b4abc37a44c8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:56.439347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 335/661] 10.222.0.64 () {56 vars in 1137 bytes} [Mon May 23 15:21:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:56.441263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daea233-80cd-483b-ae5d-e35b497f7bfa None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:56.441574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daea233-80cd-483b-ae5d-e35b497f7bfa None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:56.441835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daea233-80cd-483b-ae5d-e35b497f7bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:56.446471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fb153-da84-46ac-900f-6b9b5ad1def1 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:56.457920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4daea233-80cd-483b-ae5d-e35b497f7bfa None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:56.459127 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4daea233-80cd-483b-ae5d-e35b497f7bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:56.475736 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:56.485681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c9b23-50cc-4c57-b66a-ce982929bee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:56.486678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c9b23-50cc-4c57-b66a-ce982929bee3 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:56.487251 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c9b23-50cc-4c57-b66a-ce982929bee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:56.487611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c9b23-50cc-4c57-b66a-ce982929bee3 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:56.488624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c9b23-50cc-4c57-b66a-ce982929bee3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:56.489267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c9b23-50cc-4c57-b66a-ce982929bee3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:56.491006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c9b23-50cc-4c57-b66a-ce982929bee3 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:56.509511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 327/662] 10.222.0.64 () {56 vars in 1139 bytes} [Mon May 23 15:21:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:56.510445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c9b23-50cc-4c57-b66a-ce982929bee3 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:56.532025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 336/663] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:56.533183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daea233-80cd-483b-ae5d-e35b497f7bfa None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:56.619537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c9b23-50cc-4c57-b66a-ce982929bee3 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:56.641322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daea233-80cd-483b-ae5d-e35b497f7bfa None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:56.649857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:56.656396 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07f8aa-e016-43b2-a642-8e82ab9b2ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:56.657226 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07f8aa-e016-43b2-a642-8e82ab9b2ddd None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:56.657593 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07f8aa-e016-43b2-a642-8e82ab9b2ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:56.660719 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07f8aa-e016-43b2-a642-8e82ab9b2ddd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:56.665440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07f8aa-e016-43b2-a642-8e82ab9b2ddd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:56.666896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07f8aa-e016-43b2-a642-8e82ab9b2ddd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:56.678593 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eaa4ce-990d-4954-a7d9-04712bccd321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:56.679246 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaa4ce-990d-4954-a7d9-04712bccd321 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:56.679537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaa4ce-990d-4954-a7d9-04712bccd321 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:56.679817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaa4ce-990d-4954-a7d9-04712bccd321 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:56.680972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-ce07f8aa-e016-43b2-a642-8e82ab9b2ddd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:56.680972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:56.680972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:56.680972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:56.680972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:56.680972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:56.680972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:56.680972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:56.680972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:56.680972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:56.680972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:56.680972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:56.680972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:21:56.680972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:21:56.680972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:56.681902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:56.682865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:21:56.682865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:21:56.682865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:21:56.682865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:21:56.682865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:21:56.682865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:21:56.682865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:56.682865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:56.686770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 328/664] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:56.687548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07f8aa-e016-43b2-a642-8e82ab9b2ddd None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:56.741374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:56.760852 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eaa4ce-990d-4954-a7d9-04712bccd321 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:56.762373 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eaa4ce-990d-4954-a7d9-04712bccd321 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:56.795881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07f8aa-e016-43b2-a642-8e82ab9b2ddd None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:56.831280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b58b8-64b0-45f3-a50e-fd31cec64ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:56.831931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b58b8-64b0-45f3-a50e-fd31cec64ada None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:56.832220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b58b8-64b0-45f3-a50e-fd31cec64ada None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:56.832494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b58b8-64b0-45f3-a50e-fd31cec64ada None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:56.869098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 337/665] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:56.869510 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eaa4ce-990d-4954-a7d9-04712bccd321 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:56.878675 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:56.911388 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b58b8-64b0-45f3-a50e-fd31cec64ada None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:56.914053 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b58b8-64b0-45f3-a50e-fd31cec64ada None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:56.965185 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:56.978437 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eaa4ce-990d-4954-a7d9-04712bccd321 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:57.014508 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2ecc7-13ec-456e-bef1-9268463be680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:57.015309 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2ecc7-13ec-456e-bef1-9268463be680 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:57.015591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2ecc7-13ec-456e-bef1-9268463be680 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:57.015864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2ecc7-13ec-456e-bef1-9268463be680 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:57.023043 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 329/666] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:21:57.023961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b58b8-64b0-45f3-a50e-fd31cec64ada None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:57.031765 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2ecc7-13ec-456e-bef1-9268463be680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a8e2ecc7-13ec-456e-bef1-9268463be680 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:57.039538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:57.040492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:57.041428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:57.041428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:57.041428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:57.041428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:57.041428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:57.041428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:57.041428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:57.041428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:57.041428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:57.041428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:57.041428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:21:57.041428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:57.041428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 338/667] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:21:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:57.042147 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2ecc7-13ec-456e-bef1-9268463be680 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:57.120222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:57.135873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b58b8-64b0-45f3-a50e-fd31cec64ada None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:57.159950 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2ecc7-13ec-456e-bef1-9268463be680 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:57.170850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b2686-47a2-4a56-9e22-a4e0fed7c229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:57.171461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b2686-47a2-4a56-9e22-a4e0fed7c229 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:57.172051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b2686-47a2-4a56-9e22-a4e0fed7c229 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:57.172347 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b2686-47a2-4a56-9e22-a4e0fed7c229 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:57.173175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b2686-47a2-4a56-9e22-a4e0fed7c229 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:57.173496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b2686-47a2-4a56-9e22-a4e0fed7c229 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:57.175542 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b2686-47a2-4a56-9e22-a4e0fed7c229 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:57.187691 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 330/668] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:21:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:57.188768 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b2686-47a2-4a56-9e22-a4e0fed7c229 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:57.198498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:57.202912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2f8ad-322f-44fd-b75d-d554c4ab9f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:57.203894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2f8ad-322f-44fd-b75d-d554c4ab9f93 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:57.204334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2f8ad-322f-44fd-b75d-d554c4ab9f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:57.204785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2f8ad-322f-44fd-b75d-d554c4ab9f93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:57.216899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2f8ad-322f-44fd-b75d-d554c4ab9f93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:57.218541 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2f8ad-322f-44fd-b75d-d554c4ab9f93 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:57.237113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-62f2f8ad-322f-44fd-b75d-d554c4ab9f93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:57.237113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:57.237113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:57.237113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:57.237113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:57.237113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:57.237113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:57.237113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:57.237113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:57.237113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:57.237113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:57.237113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:57.237113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:57.237113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:57.237113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:57.238153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:57.239070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:57.239070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:57.239070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:57.239070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:57.239070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 339/669] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:57.239372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2f8ad-322f-44fd-b75d-d554c4ab9f93 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:57.300962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b2686-47a2-4a56-9e22-a4e0fed7c229 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:57.334684 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0aac9-0b16-4d14-be19-7d684d4851dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:57.335486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0aac9-0b16-4d14-be19-7d684d4851dd None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:57.335777 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0aac9-0b16-4d14-be19-7d684d4851dd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:57.336059 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0aac9-0b16-4d14-be19-7d684d4851dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:57.342616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2f8ad-322f-44fd-b75d-d554c4ab9f93 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:57.346283 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0aac9-0b16-4d14-be19-7d684d4851dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:57.347826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0aac9-0b16-4d14-be19-7d684d4851dd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:57.359648 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:57.364737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-fda0aac9-0b16-4d14-be19-7d684d4851dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:57.364737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:57.364737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:57.364737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:57.364737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:57.364737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:57.364737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:57.364737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:57.364737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:57.364737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:57.364737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:57.364737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:57.364737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:57.364737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:57.364737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:57.365670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:57.366652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:57.366652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:57.366652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:57.366652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:57.366652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 331/670] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:57.366940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0aac9-0b16-4d14-be19-7d684d4851dd None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:57.379017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c4a6a-c259-465a-8fa9-ec3f01519245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:57.379695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c4a6a-c259-465a-8fa9-ec3f01519245 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:57.379982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c4a6a-c259-465a-8fa9-ec3f01519245 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:57.380264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c4a6a-c259-465a-8fa9-ec3f01519245 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:57.381016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c4a6a-c259-465a-8fa9-ec3f01519245 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:57.381384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c4a6a-c259-465a-8fa9-ec3f01519245 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:57.382747 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c4a6a-c259-465a-8fa9-ec3f01519245 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:57.391799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 340/671] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:21:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:57.392877 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c4a6a-c259-465a-8fa9-ec3f01519245 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:57.429219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:57.470440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0aac9-0b16-4d14-be19-7d684d4851dd None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:57.490173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c4a6a-c259-465a-8fa9-ec3f01519245 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:57.504471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dccd2-ad30-4942-bb16-5acb9a9a4022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:57.505163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dccd2-ad30-4942-bb16-5acb9a9a4022 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:57.505450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dccd2-ad30-4942-bb16-5acb9a9a4022 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:57.505730 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dccd2-ad30-4942-bb16-5acb9a9a4022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:57.506556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8dccd2-ad30-4942-bb16-5acb9a9a4022 None admin] RBAC: Adding query filter params (role.id=3d6e986dd2264e3cb7e01e25dbac6f97, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=45ee7a6c342548f98e7dc73f4f71a66d) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:57.506867 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8dccd2-ad30-4942-bb16-5acb9a9a4022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:57.508290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8dccd2-ad30-4942-bb16-5acb9a9a4022 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:57.514207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 332/672] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:21:57 2022] GET /identity/v3/role_assignments?user.id=45ee7a6c342548f98e7dc73f4f71a66d&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=3d6e986dd2264e3cb7e01e25dbac6f97 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:57.515223 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528f7a6-45d1-4472-ba9e-4c7ef046bee4 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:57.515511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528f7a6-45d1-4472-ba9e-4c7ef046bee4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:57.515789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528f7a6-45d1-4472-ba9e-4c7ef046bee4 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:57.517336 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 333/673] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:21:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:21:57.517714 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528f7a6-45d1-4472-ba9e-4c7ef046bee4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:57.530319 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c5c75-fbda-4c00-9f82-194a0c085e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:57.531011 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c5c75-fbda-4c00-9f82-194a0c085e71 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:57.531296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c5c75-fbda-4c00-9f82-194a0c085e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:57.531602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c5c75-fbda-4c00-9f82-194a0c085e71 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:57.532320 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c5c75-fbda-4c00-9f82-194a0c085e71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:57.532635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c5c75-fbda-4c00-9f82-194a0c085e71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:57.534452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c5c75-fbda-4c00-9f82-194a0c085e71 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:57.548160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 341/674] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:57.550807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c5c75-fbda-4c00-9f82-194a0c085e71 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:57.614071 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:57.649694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528f7a6-45d1-4472-ba9e-4c7ef046bee4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:57.675310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:57.688591 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bb1d1-3d70-495f-bf3b-13ade177b69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:57.689232 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bb1d1-3d70-495f-bf3b-13ade177b69a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:57.689526 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bb1d1-3d70-495f-bf3b-13ade177b69a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:57.689735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c5c75-fbda-4c00-9f82-194a0c085e71 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:57.690175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bb1d1-3d70-495f-bf3b-13ade177b69a None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:57.691101 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bb1d1-3d70-495f-bf3b-13ade177b69a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:57.691416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bb1d1-3d70-495f-bf3b-13ade177b69a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:57.692814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bb1d1-3d70-495f-bf3b-13ade177b69a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:57.712816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 334/675] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:57.713908 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0c951-969d-4465-a8d3-5ada331ed078 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:57.713908 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0c951-969d-4465-a8d3-5ada331ed078 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:57.714102 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0c951-969d-4465-a8d3-5ada331ed078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:57.731686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b5d0c951-969d-4465-a8d3-5ada331ed078 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:21:57.732890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b5d0c951-969d-4465-a8d3-5ada331ed078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:21:57.734237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac96ef-a7f9-4232-92b9-7ce80f373173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:57.735276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac96ef-a7f9-4232-92b9-7ce80f373173 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:57.735387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac96ef-a7f9-4232-92b9-7ce80f373173 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:57.735566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac96ef-a7f9-4232-92b9-7ce80f373173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:57.749095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac96ef-a7f9-4232-92b9-7ce80f373173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:57.750555 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac96ef-a7f9-4232-92b9-7ce80f373173 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:57.769679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-40ac96ef-a7f9-4232-92b9-7ce80f373173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:57.769679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:57.769679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:57.769679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:57.769679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:57.769679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:57.769679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:57.769679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:57.769679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:57.769679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:57.769679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:57.769679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:57.769679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:57.769679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:57.769679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:57.770641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:57.771944 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:57.771944 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:57.771944 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:57.771944 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:57.771944 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 342/676] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:57.771944 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac96ef-a7f9-4232-92b9-7ce80f373173 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:57.811722 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 335/677] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:21:57.812395 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0c951-969d-4465-a8d3-5ada331ed078 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:57.881634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:57.890670 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac96ef-a7f9-4232-92b9-7ce80f373173 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:57.927710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ff1eb-c8d7-48a5-8a14-a5aa727dc064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:57.928533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ff1eb-c8d7-48a5-8a14-a5aa727dc064 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:57.928819 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ff1eb-c8d7-48a5-8a14-a5aa727dc064 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:57.929125 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ff1eb-c8d7-48a5-8a14-a5aa727dc064 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:57.929314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0c951-969d-4465-a8d3-5ada331ed078 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:57.936213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ff1eb-c8d7-48a5-8a14-a5aa727dc064 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:57.937699 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ff1eb-c8d7-48a5-8a14-a5aa727dc064 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:57.943873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:57.952638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-e19ff1eb-c8d7-48a5-8a14-a5aa727dc064 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:57.952638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:57.952638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:57.952638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:57.952638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:57.952638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:57.952638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:57.952638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:57.952638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:57.952638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:57.952638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:57.952638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:57.952638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:21:57.952638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:21:57.952638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:57.953628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:57.954553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:21:57.954553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:21:57.954553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:21:57.954553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:21:57.954553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:21:57.954553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:21:57.954553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:57.954553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:57.954553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 343/678] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:57.955740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ff1eb-c8d7-48a5-8a14-a5aa727dc064 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:57.969170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8f697-9e68-4b43-ac47-be29d4b714cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:57.970379 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8f697-9e68-4b43-ac47-be29d4b714cd None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:57.970836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8f697-9e68-4b43-ac47-be29d4b714cd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:57.971330 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8f697-9e68-4b43-ac47-be29d4b714cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:57.979764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8f697-9e68-4b43-ac47-be29d4b714cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:57.981748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8f697-9e68-4b43-ac47-be29d4b714cd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:57.997119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c6c8f697-9e68-4b43-ac47-be29d4b714cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:57.997119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:57.997119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:57.997119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:57.997119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:57.997119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:57.997119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:57.997119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:57.997119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:57.997119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:57.997119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:57.997119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:57.997119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:21:57.997119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:21:57.997119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:57.998229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:57.999331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:21:57.999331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:21:57.999331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:21:57.999331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:21:57.999331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:21:57.999331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:21:57.999331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:21:57.999331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:58.002958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 336/679] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:58.003959 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8f697-9e68-4b43-ac47-be29d4b714cd None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:58.064831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ff1eb-c8d7-48a5-8a14-a5aa727dc064 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:58.101928 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449d27c-b5be-4b52-be7a-c83ee73297e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:58.102599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449d27c-b5be-4b52-be7a-c83ee73297e3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:58.102894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449d27c-b5be-4b52-be7a-c83ee73297e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:58.103177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449d27c-b5be-4b52-be7a-c83ee73297e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:58.103898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e449d27c-b5be-4b52-be7a-c83ee73297e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:58.104212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e449d27c-b5be-4b52-be7a-c83ee73297e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:58.105633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e449d27c-b5be-4b52-be7a-c83ee73297e3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:58.117656 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 344/680] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:58.118836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449d27c-b5be-4b52-be7a-c83ee73297e3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:58.133740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8f697-9e68-4b43-ac47-be29d4b714cd None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:58.135082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:58.171193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7cdd2-edbf-4203-a30c-a1bbfcc78aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:58.175699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7cdd2-edbf-4203-a30c-a1bbfcc78aba None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:58.175699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7cdd2-edbf-4203-a30c-a1bbfcc78aba None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:58.175699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7cdd2-edbf-4203-a30c-a1bbfcc78aba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:58.183895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7cdd2-edbf-4203-a30c-a1bbfcc78aba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-a7f7cdd2-edbf-4203-a30c-a1bbfcc78aba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:21:58.192015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:58.192993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:58.193953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:58.193953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:58.193953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:58.193953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:58.193953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:21:58.193953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:21:58.193953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:21:58.193953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:21:58.193953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:21:58.193953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:21:58.193953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:21:58.193953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:58.193953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 337/681] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:21:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:58.194688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7cdd2-edbf-4203-a30c-a1bbfcc78aba None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:58.226036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:58.231456 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449d27c-b5be-4b52-be7a-c83ee73297e3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:58.266360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362eb28-f968-4be1-9ea3-2f6a79d30874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:58.267042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362eb28-f968-4be1-9ea3-2f6a79d30874 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:58.267335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362eb28-f968-4be1-9ea3-2f6a79d30874 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:58.267616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362eb28-f968-4be1-9ea3-2f6a79d30874 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:58.268434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362eb28-f968-4be1-9ea3-2f6a79d30874 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:58.269400 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362eb28-f968-4be1-9ea3-2f6a79d30874 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:58.270877 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362eb28-f968-4be1-9ea3-2f6a79d30874 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:58.287949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 345/682] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:21:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:58.287949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362eb28-f968-4be1-9ea3-2f6a79d30874 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:58.302761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7cdd2-edbf-4203-a30c-a1bbfcc78aba None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:58.336186 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268f588-6152-478f-92d7-b897dd1b7981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:58.336855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268f588-6152-478f-92d7-b897dd1b7981 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:58.337166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268f588-6152-478f-92d7-b897dd1b7981 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:58.337446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268f588-6152-478f-92d7-b897dd1b7981 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:58.338238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268f588-6152-478f-92d7-b897dd1b7981 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:58.338550 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268f588-6152-478f-92d7-b897dd1b7981 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:58.340269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268f588-6152-478f-92d7-b897dd1b7981 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:58.352867 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 338/683] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:21:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:58.353739 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268f588-6152-478f-92d7-b897dd1b7981 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:58.363953 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:58.393491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362eb28-f968-4be1-9ea3-2f6a79d30874 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:58.431591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ca26e-69e9-400a-bdde-cfa08c22ba53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:58.432418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ca26e-69e9-400a-bdde-cfa08c22ba53 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:58.432729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ca26e-69e9-400a-bdde-cfa08c22ba53 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:58.432978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ca26e-69e9-400a-bdde-cfa08c22ba53 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:58.440164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ca26e-69e9-400a-bdde-cfa08c22ba53 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:58.442208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ca26e-69e9-400a-bdde-cfa08c22ba53 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-805ca26e-69e9-400a-bdde-cfa08c22ba53 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:58.456240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:58.457210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:58.458213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:58.458213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:58.458213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:58.458213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:58.458213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:58.458213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:58.458213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:58.458213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:58.458213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:21:58.458213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:58.458213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 346/684] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:21:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:58.458213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ca26e-69e9-400a-bdde-cfa08c22ba53 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:58.464421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:58.473598 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268f588-6152-478f-92d7-b897dd1b7981 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:58.508811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd640d-0a46-4e0c-9a96-1bbdb3c6d8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:58.509606 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd640d-0a46-4e0c-9a96-1bbdb3c6d8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:58.509891 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd640d-0a46-4e0c-9a96-1bbdb3c6d8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:58.510406 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd640d-0a46-4e0c-9a96-1bbdb3c6d8b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:58.510941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd640d-0a46-4e0c-9a96-1bbdb3c6d8b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:58.511652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd640d-0a46-4e0c-9a96-1bbdb3c6d8b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:58.514913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd640d-0a46-4e0c-9a96-1bbdb3c6d8b6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:58.524297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 339/685] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:21:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:58.525322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd640d-0a46-4e0c-9a96-1bbdb3c6d8b6 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:58.572941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ca26e-69e9-400a-bdde-cfa08c22ba53 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:58.617633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:58.625137 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40f40f-691c-4935-948c-1df601822991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:58.626187 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40f40f-691c-4935-948c-1df601822991 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:58.626658 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40f40f-691c-4935-948c-1df601822991 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:58.627129 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40f40f-691c-4935-948c-1df601822991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:58.629143 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40f40f-691c-4935-948c-1df601822991 None admin] RBAC: Adding query filter params (role.id=3d6e986dd2264e3cb7e01e25dbac6f97, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=8750b0450bf343f590c52b7d5c68d5df) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:58.629653 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40f40f-691c-4935-948c-1df601822991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:58.631651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40f40f-691c-4935-948c-1df601822991 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:58.640911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 347/686] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:21:58 2022] GET /identity/v3/role_assignments?user.id=8750b0450bf343f590c52b7d5c68d5df&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=3d6e986dd2264e3cb7e01e25dbac6f97 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:58.642510 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40f40f-691c-4935-948c-1df601822991 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:58.689262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd640d-0a46-4e0c-9a96-1bbdb3c6d8b6 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:58.742471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bac89-bb79-4acc-8693-47d6bdaa4a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:58.743453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bac89-bb79-4acc-8693-47d6bdaa4a44 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:58.744501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bac89-bb79-4acc-8693-47d6bdaa4a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:58.744952 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bac89-bb79-4acc-8693-47d6bdaa4a44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:58.746792 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bac89-bb79-4acc-8693-47d6bdaa4a44 None admin] RBAC: Adding query filter params (role.id=3d6e986dd2264e3cb7e01e25dbac6f97, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=12ceb8a355a44b9fb29918483379cb38) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:58.747283 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bac89-bb79-4acc-8693-47d6bdaa4a44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:58.749764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bac89-bb79-4acc-8693-47d6bdaa4a44 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:58.758807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 340/687] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:21:58 2022] GET /identity/v3/role_assignments?user.id=12ceb8a355a44b9fb29918483379cb38&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=3d6e986dd2264e3cb7e01e25dbac6f97 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:58.760521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bac89-bb79-4acc-8693-47d6bdaa4a44 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:58.769902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40f40f-691c-4935-948c-1df601822991 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:58.792103 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:58.810321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324f3d1-68ba-4260-a07d-eaa12e045800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:58.811065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324f3d1-68ba-4260-a07d-eaa12e045800 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:58.811352 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324f3d1-68ba-4260-a07d-eaa12e045800 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:58.811683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324f3d1-68ba-4260-a07d-eaa12e045800 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:58.812550 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324f3d1-68ba-4260-a07d-eaa12e045800 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:58.812998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324f3d1-68ba-4260-a07d-eaa12e045800 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:58.814869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324f3d1-68ba-4260-a07d-eaa12e045800 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:58.836454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 348/688] 10.222.0.64 () {56 vars in 1135 bytes} [Mon May 23 15:21:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:58.850257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324f3d1-68ba-4260-a07d-eaa12e045800 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:58.900768 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bac89-bb79-4acc-8693-47d6bdaa4a44 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:58.953581 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:58.980937 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee8a29-1a8f-4cad-963c-e3dbd39160b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:58.981794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee8a29-1a8f-4cad-963c-e3dbd39160b0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:58.982082 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee8a29-1a8f-4cad-963c-e3dbd39160b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:58.982364 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee8a29-1a8f-4cad-963c-e3dbd39160b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:58.992121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324f3d1-68ba-4260-a07d-eaa12e045800 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:58.995806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee8a29-1a8f-4cad-963c-e3dbd39160b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:58.997313 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee8a29-1a8f-4cad-963c-e3dbd39160b0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:59.017301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-e2ee8a29-1a8f-4cad-963c-e3dbd39160b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:59.017301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:59.017301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:59.017301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:59.017301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:59.017301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:59.017301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:59.017301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:59.017301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:59.017301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:59.017301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:59.017301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:59.017301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:59.017301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:59.017301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:59.018261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:59.020415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:59.020415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:59.020415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:59.020415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:59.020415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 341/689] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:59.024874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee8a29-1a8f-4cad-963c-e3dbd39160b0 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:59.039530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85181cbf-110b-4d13-b06e-bb348986d6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:59.040288 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85181cbf-110b-4d13-b06e-bb348986d6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:59.040578 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85181cbf-110b-4d13-b06e-bb348986d6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:59.040857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85181cbf-110b-4d13-b06e-bb348986d6c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:59.103456 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:59.122905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85181cbf-110b-4d13-b06e-bb348986d6c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:59.124409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85181cbf-110b-4d13-b06e-bb348986d6c8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:59.142587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee8a29-1a8f-4cad-963c-e3dbd39160b0 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:59.184477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766e6c6-a837-40e8-925a-b7efbf886a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:59.185184 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766e6c6-a837-40e8-925a-b7efbf886a69 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:59.185475 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766e6c6-a837-40e8-925a-b7efbf886a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:59.185758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766e6c6-a837-40e8-925a-b7efbf886a69 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:59.186494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766e6c6-a837-40e8-925a-b7efbf886a69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:59.186811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766e6c6-a837-40e8-925a-b7efbf886a69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:59.188191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766e6c6-a837-40e8-925a-b7efbf886a69 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:59.200954 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 342/690] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:59.202199 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:59.206726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766e6c6-a837-40e8-925a-b7efbf886a69 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:59.239105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 349/691] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:21:59.246259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85181cbf-110b-4d13-b06e-bb348986d6c8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:59.382998 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766e6c6-a837-40e8-925a-b7efbf886a69 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:59.421370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:59.432520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a001d-b973-4335-b5cc-3b64a80773b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:59.433406 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a001d-b973-4335-b5cc-3b64a80773b9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:59.433700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a001d-b973-4335-b5cc-3b64a80773b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:59.436671 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a001d-b973-4335-b5cc-3b64a80773b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:59.443703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85181cbf-110b-4d13-b06e-bb348986d6c8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:59.451767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a001d-b973-4335-b5cc-3b64a80773b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:59.453367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a001d-b973-4335-b5cc-3b64a80773b9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-2b9a001d-b973-4335-b5cc-3b64a80773b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:21:59.474664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:59.475700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:59.476745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:59.476745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:59.476745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:21:59.476745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:21:59.476745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:21:59.476745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:21:59.476745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:21:59.476745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:21:59.476745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:21:59.476745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:21:59.483664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 343/692] 10.222.0.64 () {56 vars in 1093 bytes} [Mon May 23 15:21:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:59.490030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a001d-b973-4335-b5cc-3b64a80773b9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:59.513437 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2f014-1dd1-4ef7-8c56-57b16e288256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:59.514170 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2f014-1dd1-4ef7-8c56-57b16e288256 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:59.514465 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2f014-1dd1-4ef7-8c56-57b16e288256 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:59.514752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2f014-1dd1-4ef7-8c56-57b16e288256 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:59.534779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f2f014-1dd1-4ef7-8c56-57b16e288256 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:59.536238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f2f014-1dd1-4ef7-8c56-57b16e288256 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:59.572318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f7f2f014-1dd1-4ef7-8c56-57b16e288256 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:59.572318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:21:59.572318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:21:59.572318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:21:59.572318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:21:59.572318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:21:59.572318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:21:59.572318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:21:59.572318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:21:59.572318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:21:59.572318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:21:59.572318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:21:59.572318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:21:59.572318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:21:59.572318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:21:59.573381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:21:59.574294 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:21:59.574294 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:21:59.574294 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:21:59.574294 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:21:59.575960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 350/693] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:21:59.581016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2f014-1dd1-4ef7-8c56-57b16e288256 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:59.609460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:59.683613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a001d-b973-4335-b5cc-3b64a80773b9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:59.744026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8c9f6-e0ec-4b66-b59a-10896da377ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:59.744718 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8c9f6-e0ec-4b66-b59a-10896da377ba None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:59.745014 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8c9f6-e0ec-4b66-b59a-10896da377ba None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:59.745326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8c9f6-e0ec-4b66-b59a-10896da377ba None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:59.746238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8c9f6-e0ec-4b66-b59a-10896da377ba None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:59.746558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8c9f6-e0ec-4b66-b59a-10896da377ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:59.748007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8c9f6-e0ec-4b66-b59a-10896da377ba None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:59.767171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 344/694] 10.222.0.64 () {56 vars in 1133 bytes} [Mon May 23 15:21:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:59.773251 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8c9f6-e0ec-4b66-b59a-10896da377ba None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:21:59.793725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2f014-1dd1-4ef7-8c56-57b16e288256 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:59.823972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:21:59.876555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8c9f6-e0ec-4b66-b59a-10896da377ba None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:21:59.882689 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcccfd5-f6e6-4788-a7be-8ef00d3f61b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:59.883655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcccfd5-f6e6-4788-a7be-8ef00d3f61b8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:59.884175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcccfd5-f6e6-4788-a7be-8ef00d3f61b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:59.884661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcccfd5-f6e6-4788-a7be-8ef00d3f61b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:59.885639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcccfd5-f6e6-4788-a7be-8ef00d3f61b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:21:59.886173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcccfd5-f6e6-4788-a7be-8ef00d3f61b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:21:59.887744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcccfd5-f6e6-4788-a7be-8ef00d3f61b8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:21:59.911357 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32d0e2-25a7-4842-af15-afbe5bc19c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:21:59.912026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32d0e2-25a7-4842-af15-afbe5bc19c61 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:21:59.912320 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32d0e2-25a7-4842-af15-afbe5bc19c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:21:59.912605 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32d0e2-25a7-4842-af15-afbe5bc19c61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:21:59.919607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 351/695] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:21:59.927553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcccfd5-f6e6-4788-a7be-8ef00d3f61b8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:00.028218 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32d0e2-25a7-4842-af15-afbe5bc19c61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:00.029811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32d0e2-25a7-4842-af15-afbe5bc19c61 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:00.037924 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:00.083768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcccfd5-f6e6-4788-a7be-8ef00d3f61b8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:00.095000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:00.133935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c448ce0-aa53-44e0-b94a-a4e3e54deafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:00.135298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c448ce0-aa53-44e0-b94a-a4e3e54deafd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:00.135773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c448ce0-aa53-44e0-b94a-a4e3e54deafd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:00.136422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c448ce0-aa53-44e0-b94a-a4e3e54deafd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:00.144781 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c448ce0-aa53-44e0-b94a-a4e3e54deafd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:00.146292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c448ce0-aa53-44e0-b94a-a4e3e54deafd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:00.163256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-4c448ce0-aa53-44e0-b94a-a4e3e54deafd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:00.163256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:00.163256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:00.163256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:00.163256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:00.163256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:00.163256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:00.163256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:00.163256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:00.163256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:00.163256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:00.163256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:00.163256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:00.163256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:00.163256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:00.164238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:00.165189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:00.165189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:00.165189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:00.165189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:00.165189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:00.165189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:00.165189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:00.165189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:00.166394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 345/697] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:21:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:22:00.166601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 352/697] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:21:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:00.168227 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a121045-f138-4746-a30d-46dccb31af93 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:00.168462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c448ce0-aa53-44e0-b94a-a4e3e54deafd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:00.168948 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a121045-f138-4746-a30d-46dccb31af93 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:00.169464 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a121045-f138-4746-a30d-46dccb31af93 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:00.171277 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 346/698] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:00.172205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a121045-f138-4746-a30d-46dccb31af93 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:00.288314 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c448ce0-aa53-44e0-b94a-a4e3e54deafd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:00.316566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a121045-f138-4746-a30d-46dccb31af93 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:00.317405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:00.326193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acbcb8-d1c2-40a3-8c2b-241ccf2f9525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:00.327369 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acbcb8-d1c2-40a3-8c2b-241ccf2f9525 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:00.327485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acbcb8-d1c2-40a3-8c2b-241ccf2f9525 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:00.329914 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acbcb8-d1c2-40a3-8c2b-241ccf2f9525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:00.339416 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58acbcb8-d1c2-40a3-8c2b-241ccf2f9525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:00.340870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58acbcb8-d1c2-40a3-8c2b-241ccf2f9525 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:00.357477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37caa240-198c-47d0-a49f-112a87059e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:00.358441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caa240-198c-47d0-a49f-112a87059e53 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:00.358963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caa240-198c-47d0-a49f-112a87059e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:00.359456 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caa240-198c-47d0-a49f-112a87059e53 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:00.360453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37caa240-198c-47d0-a49f-112a87059e53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:00.360571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-58acbcb8-d1c2-40a3-8c2b-241ccf2f9525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:00.360571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:00.360571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:00.360571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:00.360571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:00.360571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:00.360571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:00.360571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:00.360571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:00.360571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:00.360571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:00.360571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:00.360571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:00.360571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:00.360571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:00.361461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:00.362946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:00.362946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:00.362946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:00.362946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:00.364003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37caa240-198c-47d0-a49f-112a87059e53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:00.365708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37caa240-198c-47d0-a49f-112a87059e53 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:00.366900 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 353/699] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:00.368393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea44d7-765d-4887-b84a-ba51afb61fe4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:00.368680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea44d7-765d-4887-b84a-ba51afb61fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:00.368955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea44d7-765d-4887-b84a-ba51afb61fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:00.379737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 347/700] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:00.380934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37caa240-198c-47d0-a49f-112a87059e53 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:00.386524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e6ea44d7-765d-4887-b84a-ba51afb61fe4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:00.387702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e6ea44d7-765d-4887-b84a-ba51afb61fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:00.392995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:00.461412 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 354/701] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:22:00.462265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea44d7-765d-4887-b84a-ba51afb61fe4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:00.490996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37caa240-198c-47d0-a49f-112a87059e53 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:00.528626 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4c451-a1bb-4c78-a173-ec8f4db55698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:00.529746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4c451-a1bb-4c78-a173-ec8f4db55698 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:00.530245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4c451-a1bb-4c78-a173-ec8f4db55698 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:00.530868 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4c451-a1bb-4c78-a173-ec8f4db55698 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:00.531899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4c451-a1bb-4c78-a173-ec8f4db55698 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:00.532525 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4c451-a1bb-4c78-a173-ec8f4db55698 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:00.534187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4c451-a1bb-4c78-a173-ec8f4db55698 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:00.547787 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 348/702] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:00.548860 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4c451-a1bb-4c78-a173-ec8f4db55698 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:00.572418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea44d7-765d-4887-b84a-ba51afb61fe4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:00.583036 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:00.609029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ab692-7e59-4af2-8514-4012ee213002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:00.620452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ab692-7e59-4af2-8514-4012ee213002 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:00.620452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ab692-7e59-4af2-8514-4012ee213002 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:00.620452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ab692-7e59-4af2-8514-4012ee213002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:00.620452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ab692-7e59-4af2-8514-4012ee213002 None admin] RBAC: Adding query filter params (role.id=3d6e986dd2264e3cb7e01e25dbac6f97, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=f8c79c36c7204368816593f0e473191b) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:00.620452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ab692-7e59-4af2-8514-4012ee213002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:00.620452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ab692-7e59-4af2-8514-4012ee213002 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:00.620888 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 355/703] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:22:00 2022] GET /identity/v3/role_assignments?user.id=f8c79c36c7204368816593f0e473191b&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=3d6e986dd2264e3cb7e01e25dbac6f97 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:00.620888 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ab692-7e59-4af2-8514-4012ee213002 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:00.640155 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:00.700098 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4c451-a1bb-4c78-a173-ec8f4db55698 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:00.767019 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc117edb-81d9-4d6f-b0c8-c51f4501b867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:00.768784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc117edb-81d9-4d6f-b0c8-c51f4501b867 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:00.769084 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc117edb-81d9-4d6f-b0c8-c51f4501b867 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:00.769520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc117edb-81d9-4d6f-b0c8-c51f4501b867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:00.784468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc117edb-81d9-4d6f-b0c8-c51f4501b867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:00.786118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc117edb-81d9-4d6f-b0c8-c51f4501b867 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:00.796773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ab692-7e59-4af2-8514-4012ee213002 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:00.810806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-fc117edb-81d9-4d6f-b0c8-c51f4501b867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:00.810806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:00.810806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:00.810806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:00.810806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:00.810806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:00.810806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:00.810806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:00.810806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:00.810806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:00.810806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:00.810806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:00.810806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:00.810806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:00.810806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:00.811767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:00.812686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:00.812686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:00.812686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:00.812686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:00.814492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 349/704] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:00.821082 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc117edb-81d9-4d6f-b0c8-c51f4501b867 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:00.875635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82810d6a-31db-48d2-a827-a624d6e699e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:00.876457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82810d6a-31db-48d2-a827-a624d6e699e2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:00.876789 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82810d6a-31db-48d2-a827-a624d6e699e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:00.877086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82810d6a-31db-48d2-a827-a624d6e699e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:00.890683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82810d6a-31db-48d2-a827-a624d6e699e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:00.896697 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82810d6a-31db-48d2-a827-a624d6e699e2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:00.914565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-82810d6a-31db-48d2-a827-a624d6e699e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:00.914565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:00.914565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:00.914565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:00.914565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:00.914565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:00.914565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:00.914565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:00.914565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:00.914565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:00.914565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:00.914565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:00.914565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:00.914565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:00.914565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:00.915533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:00.916441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:00.916441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:00.916441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:00.916441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:00.916441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:00.916441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:00.916441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:00.916441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:00.916889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 356/705] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:00.922291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82810d6a-31db-48d2-a827-a624d6e699e2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:00.958195 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:00.972183 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc117edb-81d9-4d6f-b0c8-c51f4501b867 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:00.993937 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:01.009156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a0468-3b58-4214-8c4a-9a8cd72aac1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:01.009900 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a0468-3b58-4214-8c4a-9a8cd72aac1e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:01.010194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a0468-3b58-4214-8c4a-9a8cd72aac1e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:01.010483 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a0468-3b58-4214-8c4a-9a8cd72aac1e None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:01.011220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a0468-3b58-4214-8c4a-9a8cd72aac1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:01.011538 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a0468-3b58-4214-8c4a-9a8cd72aac1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:01.012930 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a0468-3b58-4214-8c4a-9a8cd72aac1e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:01.032348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 350/706] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:01.035291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a0468-3b58-4214-8c4a-9a8cd72aac1e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:01.059310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82810d6a-31db-48d2-a827-a624d6e699e2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:01.102543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff7843-a48c-4863-9adc-e729da76442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:01.103705 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff7843-a48c-4863-9adc-e729da76442a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:01.104237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff7843-a48c-4863-9adc-e729da76442a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:01.104725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff7843-a48c-4863-9adc-e729da76442a None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:01.105867 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff7843-a48c-4863-9adc-e729da76442a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:01.106478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff7843-a48c-4863-9adc-e729da76442a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:01.108204 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff7843-a48c-4863-9adc-e729da76442a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:01.122653 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 357/707] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:01.129528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff7843-a48c-4863-9adc-e729da76442a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:01.152305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a0468-3b58-4214-8c4a-9a8cd72aac1e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:01.187906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3455c21-0094-4835-b3bb-26d21cdc8225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:01.188586 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3455c21-0094-4835-b3bb-26d21cdc8225 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:01.188933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3455c21-0094-4835-b3bb-26d21cdc8225 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:01.189253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3455c21-0094-4835-b3bb-26d21cdc8225 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:01.189966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3455c21-0094-4835-b3bb-26d21cdc8225 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:01.192054 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3455c21-0094-4835-b3bb-26d21cdc8225 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:01.242666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:01.251719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff7843-a48c-4863-9adc-e729da76442a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:01.273124 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 351/708] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:22:01 2022] POST /identity/v3/users => generated 254 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:22:01.275064 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090978f-8121-4f9f-beb2-a9ae704e4e15 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:01.275463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090978f-8121-4f9f-beb2-a9ae704e4e15 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:01.275850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090978f-8121-4f9f-beb2-a9ae704e4e15 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:01.277632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 352/709] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:01.285926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf198d8-bff5-45f4-b1ec-5af13c4d6201 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:01.286207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf198d8-bff5-45f4-b1ec-5af13c4d6201 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:01.286490 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf198d8-bff5-45f4-b1ec-5af13c4d6201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:01.292518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583657f5-492c-4d08-b661-3c3e0f773226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:01.293679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583657f5-492c-4d08-b661-3c3e0f773226 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:01.294157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583657f5-492c-4d08-b661-3c3e0f773226 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:01.294621 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583657f5-492c-4d08-b661-3c3e0f773226 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:01.295821 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583657f5-492c-4d08-b661-3c3e0f773226 None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=8da96bed73794fee8f719ec904936164) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:01.296423 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583657f5-492c-4d08-b661-3c3e0f773226 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:01.298542 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583657f5-492c-4d08-b661-3c3e0f773226 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:01.306634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 358/710] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:22:01 2022] GET /identity/v3/role_assignments?user.id=8da96bed73794fee8f719ec904936164&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:01.309978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fbf198d8-bff5-45f4-b1ec-5af13c4d6201 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:01.311237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fbf198d8-bff5-45f4-b1ec-5af13c4d6201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:01.350389 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:01.395985 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 353/711] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:22:01.437534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583657f5-492c-4d08-b661-3c3e0f773226 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:01.542729 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8203727-162a-419e-bbc0-b0fdddbf0c96 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:01.542945 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8203727-162a-419e-bbc0-b0fdddbf0c96 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:01.543111 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8203727-162a-419e-bbc0-b0fdddbf0c96 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:01.544675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 354/712] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:01.551439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce793680-2df7-4594-bdc6-ac97ae3c65cb None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:01.551969 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce793680-2df7-4594-bdc6-ac97ae3c65cb None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:01.552653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce793680-2df7-4594-bdc6-ac97ae3c65cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:01.577920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ce793680-2df7-4594-bdc6-ac97ae3c65cb None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:01.579145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ce793680-2df7-4594-bdc6-ac97ae3c65cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:01.586738 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583657f5-492c-4d08-b661-3c3e0f773226 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:01.635259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfe7cf-87e5-468c-b77b-d37ffde7867a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:01.636110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfe7cf-87e5-468c-b77b-d37ffde7867a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:01.636402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfe7cf-87e5-468c-b77b-d37ffde7867a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:01.636679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfe7cf-87e5-468c-b77b-d37ffde7867a None admin] PATH_INFO: `/v3/services/network` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:01.637385 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfe7cf-87e5-468c-b77b-d37ffde7867a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:01.638761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfe7cf-87e5-468c-b77b-d37ffde7867a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-83dfe7cf-87e5-468c-b77b-d37ffde7867a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:01.655461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:01.656549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:01.657503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:01.657503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:01.657503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:01.657503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:01.657503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:01.657503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:01.657503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:01.657503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:01.657503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:01.657503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:01.657503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:01.657503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:01.657503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:22:01.657503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:01.657503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 359/713] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:01 2022] GET /identity/v3/services/network => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:01.664494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfe7cf-87e5-468c-b77b-d37ffde7867a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:01.676830 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 355/714] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:22:01.704220 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:01.707719 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce793680-2df7-4594-bdc6-ac97ae3c65cb None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:01.821278 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:01.828295 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce793680-2df7-4594-bdc6-ac97ae3c65cb None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:01.868475 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62a2ea-c652-4ff4-969a-0edf291fbebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:01.870140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62a2ea-c652-4ff4-969a-0edf291fbebf None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:01.870641 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62a2ea-c652-4ff4-969a-0edf291fbebf None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:01.871145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62a2ea-c652-4ff4-969a-0edf291fbebf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:01.872010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62a2ea-c652-4ff4-969a-0edf291fbebf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:01.873645 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62a2ea-c652-4ff4-969a-0edf291fbebf None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-be62a2ea-c652-4ff4-969a-0edf291fbebf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:01.890967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:01.891983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:01.892844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:01.892844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:01.892844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:01.892844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:01.892844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:01.892844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:01.892844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:01.892844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:01.892844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:01.892844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:01.892844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:01.892844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:01.892844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:01.892844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:22:01.892844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:01.897800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 356/715] 10.222.0.64 () {56 vars in 1129 bytes} [Mon May 23 15:22:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:01.904049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62a2ea-c652-4ff4-969a-0edf291fbebf None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:01.947055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfe7cf-87e5-468c-b77b-d37ffde7867a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:02.019238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62a2ea-c652-4ff4-969a-0edf291fbebf None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:02.061752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53b188-c76b-41c6-a3b3-6fae2827381e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:02.063282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53b188-c76b-41c6-a3b3-6fae2827381e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:02.063775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53b188-c76b-41c6-a3b3-6fae2827381e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:02.064259 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53b188-c76b-41c6-a3b3-6fae2827381e None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:02.065419 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53b188-c76b-41c6-a3b3-6fae2827381e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:02.066086 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53b188-c76b-41c6-a3b3-6fae2827381e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:02.067778 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53b188-c76b-41c6-a3b3-6fae2827381e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:02.070387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40834d-2148-4b6d-b806-15c4cb40a8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:02.071077 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40834d-2148-4b6d-b806-15c4cb40a8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:02.071369 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40834d-2148-4b6d-b806-15c4cb40a8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:02.071763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40834d-2148-4b6d-b806-15c4cb40a8a5 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:02.074233 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 357/716] 10.222.0.64 () {56 vars in 1124 bytes} [Mon May 23 15:22:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:02.079833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53b188-c76b-41c6-a3b3-6fae2827381e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:02.081336 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40834d-2148-4b6d-b806-15c4cb40a8a5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:02.082535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40834d-2148-4b6d-b806-15c4cb40a8a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:02.084153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40834d-2148-4b6d-b806-15c4cb40a8a5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:02.099898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 360/717] 10.222.0.64 () {56 vars in 1112 bytes} [Mon May 23 15:22:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:02.104956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40834d-2148-4b6d-b806-15c4cb40a8a5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:02.134350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:02.238169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:02.255646 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40834d-2148-4b6d-b806-15c4cb40a8a5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:02.294666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7542a25-7593-4031-9912-babcb9bbc4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:02.295853 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7542a25-7593-4031-9912-babcb9bbc4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:02.296382 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7542a25-7593-4031-9912-babcb9bbc4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:02.296881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7542a25-7593-4031-9912-babcb9bbc4a4 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:02.297847 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7542a25-7593-4031-9912-babcb9bbc4a4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:02.298358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7542a25-7593-4031-9912-babcb9bbc4a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:02.301112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7542a25-7593-4031-9912-babcb9bbc4a4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:02.308857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 361/718] 10.222.0.64 () {56 vars in 1112 bytes} [Mon May 23 15:22:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:02.329787 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53b188-c76b-41c6-a3b3-6fae2827381e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:02.376386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ccf07-a130-4c86-92b6-f9a0cd8ff5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:02.377110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ccf07-a130-4c86-92b6-f9a0cd8ff5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:02.377403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ccf07-a130-4c86-92b6-f9a0cd8ff5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:02.377791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ccf07-a130-4c86-92b6-f9a0cd8ff5b4 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:02.378551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ccf07-a130-4c86-92b6-f9a0cd8ff5b4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:02.378849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ccf07-a130-4c86-92b6-f9a0cd8ff5b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:02.380278 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ccf07-a130-4c86-92b6-f9a0cd8ff5b4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:02.389266 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 358/719] 10.222.0.64 () {56 vars in 1124 bytes} [Mon May 23 15:22:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:03.560059 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b3658-5fe2-4297-8251-15397e664ed6 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:03.560615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b3658-5fe2-4297-8251-15397e664ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:03.560615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b3658-5fe2-4297-8251-15397e664ed6 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:03.562250 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 362/720] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:03.568370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e9624-aabf-49b5-a516-64225265bd9d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:03.569240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e9624-aabf-49b5-a516-64225265bd9d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:03.569748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e9624-aabf-49b5-a516-64225265bd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:03.593475 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fd2e9624-aabf-49b5-a516-64225265bd9d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:03.595213 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fd2e9624-aabf-49b5-a516-64225265bd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:03.691338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 359/721] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:22:03.740152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b3658-5fe2-4297-8251-15397e664ed6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:03.865858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b3658-5fe2-4297-8251-15397e664ed6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:03.930283 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548b216-3ca0-4a23-a942-08c75a7e3710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:03.931123 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548b216-3ca0-4a23-a942-08c75a7e3710 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:03.931415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548b216-3ca0-4a23-a942-08c75a7e3710 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:03.931692 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548b216-3ca0-4a23-a942-08c75a7e3710 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:03.948502 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548b216-3ca0-4a23-a942-08c75a7e3710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-8548b216-3ca0-4a23-a942-08c75a7e3710 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:03.956833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:03.957902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:03.958758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:03.958758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:03.958758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:03.958758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:03.958758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:03.958758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:03.958758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:03.958758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:03.958758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:03.958758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:03.958758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:03.958758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:03.964133 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e9624-aabf-49b5-a516-64225265bd9d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:03.965446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 363/722] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:22:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:04.003120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:04.138907 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e9624-aabf-49b5-a516-64225265bd9d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:04.146489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6e7b3-a45d-4196-9977-d2a490adc3a6 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:04.146994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6e7b3-a45d-4196-9977-d2a490adc3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:04.147421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6e7b3-a45d-4196-9977-d2a490adc3a6 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:04.148940 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 364/723] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:04.154214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fb028-d435-4300-861f-f9eac584c122 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:04.154533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fb028-d435-4300-861f-f9eac584c122 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:04.154920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fb028-d435-4300-861f-f9eac584c122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:04.181207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc60e8-3d2f-449c-afe4-826f5c068ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:04.182166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc60e8-3d2f-449c-afe4-826f5c068ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:04.182608 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc60e8-3d2f-449c-afe4-826f5c068ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:04.183040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc60e8-3d2f-449c-afe4-826f5c068ba8 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:04.183940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc60e8-3d2f-449c-afe4-826f5c068ba8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:04.184468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc60e8-3d2f-449c-afe4-826f5c068ba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:04.186071 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc60e8-3d2f-449c-afe4-826f5c068ba8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:04.188489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-571fb028-d435-4300-861f-f9eac584c122 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:04.189713 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-571fb028-d435-4300-861f-f9eac584c122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:04.192047 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:04.195801 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 360/724] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:22:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:04.200873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc60e8-3d2f-449c-afe4-826f5c068ba8 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:04.273593 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 365/725] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:22:04.280746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dde73d-07ca-450b-bd6f-32dba871e549 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:04.281253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dde73d-07ca-450b-bd6f-32dba871e549 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:04.281689 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dde73d-07ca-450b-bd6f-32dba871e549 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:04.283727 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 366/726] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:04.292485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bfb95-ae52-4139-b091-e250c15da9c7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:04.293209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bfb95-ae52-4139-b091-e250c15da9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:04.293681 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bfb95-ae52-4139-b091-e250c15da9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:04.297421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:04.321323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9b4bfb95-ae52-4139-b091-e250c15da9c7 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:04.322516 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9b4bfb95-ae52-4139-b091-e250c15da9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:04.327391 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc60e8-3d2f-449c-afe4-826f5c068ba8 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:04.370318 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dd105-1c3c-499b-a9a5-07b226ba740c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:04.371791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dd105-1c3c-499b-a9a5-07b226ba740c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:04.372239 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dd105-1c3c-499b-a9a5-07b226ba740c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:04.373114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dd105-1c3c-499b-a9a5-07b226ba740c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:04.386380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dd105-1c3c-499b-a9a5-07b226ba740c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:04.388512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dd105-1c3c-499b-a9a5-07b226ba740c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:04.400905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 367/727] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:22:04.402102 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bfb95-ae52-4139-b091-e250c15da9c7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:04.411726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-b29dd105-1c3c-499b-a9a5-07b226ba740c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:04.411726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:04.411726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:04.411726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:04.411726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:04.411726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:04.411726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:04.411726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:04.411726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:04.411726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:04.411726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:04.411726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:04.411726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:04.411726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:04.411726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:04.412678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:04.413682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:04.413682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:04.413682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:04.413682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:04.414689 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 361/728] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:04.424022 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dd105-1c3c-499b-a9a5-07b226ba740c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:04.479825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:04.514960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bfb95-ae52-4139-b091-e250c15da9c7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:04.532662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dd105-1c3c-499b-a9a5-07b226ba740c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:04.552741 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fffa04-fe2a-42c4-8add-33e2dfb619a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:04.553603 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fffa04-fe2a-42c4-8add-33e2dfb619a0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:04.553895 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fffa04-fe2a-42c4-8add-33e2dfb619a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:04.554164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fffa04-fe2a-42c4-8add-33e2dfb619a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:04.565844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:04.566695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fffa04-fe2a-42c4-8add-33e2dfb619a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:04.571790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647c834-d5e5-4443-a5e3-c5f4abd54e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:04.572577 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647c834-d5e5-4443-a5e3-c5f4abd54e07 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:04.577997 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647c834-d5e5-4443-a5e3-c5f4abd54e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:04.577997 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647c834-d5e5-4443-a5e3-c5f4abd54e07 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:04.577997 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647c834-d5e5-4443-a5e3-c5f4abd54e07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:04.577997 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647c834-d5e5-4443-a5e3-c5f4abd54e07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d8fffa04-fe2a-42c4-8add-33e2dfb619a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:04.578307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:04.579226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:04.580323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:04.580323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:04.580323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:04.580323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:04.580323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:04.580323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:04.580323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:04.580323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:04.580323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:04.580323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:04.580323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:22:04.580323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:04.581327 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647c834-d5e5-4443-a5e3-c5f4abd54e07 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:04.581441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 368/729] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:22:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:04.581441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fffa04-fe2a-42c4-8add-33e2dfb619a0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:04.591635 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 362/730] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:04.592659 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647c834-d5e5-4443-a5e3-c5f4abd54e07 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:04.757425 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fffa04-fe2a-42c4-8add-33e2dfb619a0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:04.761289 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647c834-d5e5-4443-a5e3-c5f4abd54e07 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:04.785777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:04.794709 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfe953-9212-402c-8c73-210d7cb32529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:04.796905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfe953-9212-402c-8c73-210d7cb32529 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:04.796905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfe953-9212-402c-8c73-210d7cb32529 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:04.796905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfe953-9212-402c-8c73-210d7cb32529 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:04.797204 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5005a-a91a-4d1e-91f8-e5445e3686ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:04.798024 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5005a-a91a-4d1e-91f8-e5445e3686ac None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:04.798451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5005a-a91a-4d1e-91f8-e5445e3686ac None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:04.798870 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5005a-a91a-4d1e-91f8-e5445e3686ac None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:04.799821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5005a-a91a-4d1e-91f8-e5445e3686ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:04.800346 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5005a-a91a-4d1e-91f8-e5445e3686ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:04.802204 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5005a-a91a-4d1e-91f8-e5445e3686ac None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:04.808834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfe953-9212-402c-8c73-210d7cb32529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:04.811770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 363/731] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:22:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:04.812807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5005a-a91a-4d1e-91f8-e5445e3686ac None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6adfe953-9212-402c-8c73-210d7cb32529 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:04.818054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:04.819191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:04.820040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:04.820040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:04.820040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:04.820040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:04.820040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:04.820040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:04.820040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:04.820040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:04.820040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:04.820040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:04.820040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:04.820040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:04.821428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 369/732] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:22:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:04.826372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfe953-9212-402c-8c73-210d7cb32529 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:04.883801 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:04.939192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5005a-a91a-4d1e-91f8-e5445e3686ac None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:04.944847 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfe953-9212-402c-8c73-210d7cb32529 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:04.976891 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4018452-61a8-4c5a-92a3-09696c92ff5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:04.978043 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4018452-61a8-4c5a-92a3-09696c92ff5d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:04.978528 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4018452-61a8-4c5a-92a3-09696c92ff5d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:04.979020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4018452-61a8-4c5a-92a3-09696c92ff5d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:04.980900 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975f6f6-03b6-48b8-b4b0-41c87507768a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:04.981746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975f6f6-03b6-48b8-b4b0-41c87507768a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:04.982032 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975f6f6-03b6-48b8-b4b0-41c87507768a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:04.982311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975f6f6-03b6-48b8-b4b0-41c87507768a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:04.988444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4018452-61a8-4c5a-92a3-09696c92ff5d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:04.990297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4018452-61a8-4c5a-92a3-09696c92ff5d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:04.993625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6975f6f6-03b6-48b8-b4b0-41c87507768a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:04.994734 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6975f6f6-03b6-48b8-b4b0-41c87507768a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-f4018452-61a8-4c5a-92a3-09696c92ff5d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:22:05.007099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:05.008129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:05.009132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:05.009132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:05.009132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:22:05.009132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:22:05.009132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:22:05.009132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:22:05.009132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:22:05.009132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:22:05.009132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:22:05.009132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:05.016300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6975f6f6-03b6-48b8-b4b0-41c87507768a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:05.016300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:05.016300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:05.016300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:05.016300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:05.016300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:05.016300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:05.016300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:05.016300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:05.016300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:05.016300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:05.016300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:05.016300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:05.016300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:05.016300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:05.017311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:05.018513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:05.018513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:05.018513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:05.018513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:05.018513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 370/734] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:05.018513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975f6f6-03b6-48b8-b4b0-41c87507768a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:05.019456 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 364/734] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:22:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:05.020802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4018452-61a8-4c5a-92a3-09696c92ff5d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:05.064268 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:05.124651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975f6f6-03b6-48b8-b4b0-41c87507768a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:05.136400 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4018452-61a8-4c5a-92a3-09696c92ff5d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:05.154247 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:05.159207 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7038912-0359-435a-a88a-294aad25ce24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:05.160270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7038912-0359-435a-a88a-294aad25ce24 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.160559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7038912-0359-435a-a88a-294aad25ce24 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.160841 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7038912-0359-435a-a88a-294aad25ce24 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.161619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7038912-0359-435a-a88a-294aad25ce24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:05.164777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7038912-0359-435a-a88a-294aad25ce24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:05.164777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7038912-0359-435a-a88a-294aad25ce24 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:05.171883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 371/735] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:22:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:05.172699 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7038912-0359-435a-a88a-294aad25ce24 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:05.182240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242f129-459d-45ef-8ebf-d01c74bf8c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:05.183299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242f129-459d-45ef-8ebf-d01c74bf8c8f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.183884 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242f129-459d-45ef-8ebf-d01c74bf8c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.184910 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242f129-459d-45ef-8ebf-d01c74bf8c8f None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.186105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242f129-459d-45ef-8ebf-d01c74bf8c8f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:05.187015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242f129-459d-45ef-8ebf-d01c74bf8c8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:05.189297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242f129-459d-45ef-8ebf-d01c74bf8c8f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:05.212458 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 365/736] 10.222.0.64 () {56 vars in 1135 bytes} [Mon May 23 15:22:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:05.214000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf96ac5-68e8-4b9c-bf4e-17953b560289 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.214296 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf96ac5-68e8-4b9c-bf4e-17953b560289 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.214574 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf96ac5-68e8-4b9c-bf4e-17953b560289 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.215914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 366/737] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:05.217824 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf96ac5-68e8-4b9c-bf4e-17953b560289 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:05.278610 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7038912-0359-435a-a88a-294aad25ce24 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:05.280027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:05.312501 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9acc9-6fc0-4873-961f-1b3527b2031f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:05.313189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9acc9-6fc0-4873-961f-1b3527b2031f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.313480 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9acc9-6fc0-4873-961f-1b3527b2031f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.313761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9acc9-6fc0-4873-961f-1b3527b2031f None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.314481 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9acc9-6fc0-4873-961f-1b3527b2031f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:05.314792 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9acc9-6fc0-4873-961f-1b3527b2031f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:05.316197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9acc9-6fc0-4873-961f-1b3527b2031f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:05.317115 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf96ac5-68e8-4b9c-bf4e-17953b560289 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:05.327780 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 372/738] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:05.329114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684be2d1-7a01-4e41-8ca6-f3468320d50c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.329408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684be2d1-7a01-4e41-8ca6-f3468320d50c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.329688 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684be2d1-7a01-4e41-8ca6-f3468320d50c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.331593 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 373/739] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:05.332155 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684be2d1-7a01-4e41-8ca6-f3468320d50c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:05.354369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971d22d-1949-46b2-b9d6-74fd1853d3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:05.355162 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971d22d-1949-46b2-b9d6-74fd1853d3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.355446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971d22d-1949-46b2-b9d6-74fd1853d3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.355721 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971d22d-1949-46b2-b9d6-74fd1853d3a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.365674 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971d22d-1949-46b2-b9d6-74fd1853d3a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:05.367108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971d22d-1949-46b2-b9d6-74fd1853d3a4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:05.384762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-1971d22d-1949-46b2-b9d6-74fd1853d3a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:05.384762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:05.384762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:05.384762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:05.384762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:05.384762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:05.384762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:05.384762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:05.384762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:05.384762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:05.384762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:05.384762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:05.384762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:05.384762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:05.384762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:05.385717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:05.387076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:05.387076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:05.387076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:05.387076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:05.387076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 367/740] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:05.387452 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f2d5b-8806-458d-b4e8-b3097bf60e68 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.387452 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f2d5b-8806-458d-b4e8-b3097bf60e68 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.387665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f2d5b-8806-458d-b4e8-b3097bf60e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.404089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e82f2d5b-8806-458d-b4e8-b3097bf60e68 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:05.405294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e82f2d5b-8806-458d-b4e8-b3097bf60e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:05.422037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:05.435321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684be2d1-7a01-4e41-8ca6-f3468320d50c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:05.469047 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a2ec9-a662-4070-b4ec-5a498da48614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:05.469720 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a2ec9-a662-4070-b4ec-5a498da48614 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.470011 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a2ec9-a662-4070-b4ec-5a498da48614 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.470294 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a2ec9-a662-4070-b4ec-5a498da48614 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.472758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 368/741] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:22:05.473611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f2d5b-8806-458d-b4e8-b3097bf60e68 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:05.504039 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:05.541483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a2ec9-a662-4070-b4ec-5a498da48614 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:05.542865 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a2ec9-a662-4070-b4ec-5a498da48614 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:05.576869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f2d5b-8806-458d-b4e8-b3097bf60e68 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:05.611322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baedff6-2641-41f4-b24d-40860e744297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:05.612701 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baedff6-2641-41f4-b24d-40860e744297 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.612975 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baedff6-2641-41f4-b24d-40860e744297 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.613276 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baedff6-2641-41f4-b24d-40860e744297 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.620281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baedff6-2641-41f4-b24d-40860e744297 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:05.621750 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baedff6-2641-41f4-b24d-40860e744297 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-2baedff6-2641-41f4-b24d-40860e744297 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:22:05.636495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:05.637501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:05.638471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:05.638471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:05.638471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:22:05.638471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:22:05.638471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:22:05.638471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:22:05.638471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:22:05.638471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:22:05.638471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:22:05.638471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:05.638471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 369/742] 10.222.0.64 () {56 vars in 1117 bytes} [Mon May 23 15:22:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:05.639749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb16df1-0d7b-4d9c-9cc3-788ffcee0b11 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.639749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb16df1-0d7b-4d9c-9cc3-788ffcee0b11 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.639749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb16df1-0d7b-4d9c-9cc3-788ffcee0b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.649101 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 374/743] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:22:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:22:05.649101 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a2ec9-a662-4070-b4ec-5a498da48614 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:05.656133 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0fb16df1-0d7b-4d9c-9cc3-788ffcee0b11 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:05.657375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0fb16df1-0d7b-4d9c-9cc3-788ffcee0b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:05.667373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:05.712492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:05.725587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 370/744] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:22:05.726424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb16df1-0d7b-4d9c-9cc3-788ffcee0b11 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:05.754915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a2ec9-a662-4070-b4ec-5a498da48614 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:05.789643 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fb3c8-074c-4f82-92f8-f869ea87861d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:05.790303 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fb3c8-074c-4f82-92f8-f869ea87861d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.790589 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fb3c8-074c-4f82-92f8-f869ea87861d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.790870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fb3c8-074c-4f82-92f8-f869ea87861d None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.791649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fb3c8-074c-4f82-92f8-f869ea87861d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:05.791910 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fb3c8-074c-4f82-92f8-f869ea87861d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:05.793291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fb3c8-074c-4f82-92f8-f869ea87861d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:05.804633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 375/745] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:05.805466 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fb3c8-074c-4f82-92f8-f869ea87861d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:05.832113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb16df1-0d7b-4d9c-9cc3-788ffcee0b11 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:05.866027 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a246a5b-3d3e-4df2-a8df-b31a4d1b1156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:05.866702 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a246a5b-3d3e-4df2-a8df-b31a4d1b1156 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.866986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a246a5b-3d3e-4df2-a8df-b31a4d1b1156 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.867265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a246a5b-3d3e-4df2-a8df-b31a4d1b1156 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.867991 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a246a5b-3d3e-4df2-a8df-b31a4d1b1156 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:05.869407 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a246a5b-3d3e-4df2-a8df-b31a4d1b1156 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:05.904602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fb3c8-074c-4f82-92f8-f869ea87861d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:05.926884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:05.938140 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0114923-3f60-470b-8e8d-adca2f88e017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:05.938806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0114923-3f60-470b-8e8d-adca2f88e017 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:05.939093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0114923-3f60-470b-8e8d-adca2f88e017 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:05.939372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0114923-3f60-470b-8e8d-adca2f88e017 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:05.940201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0114923-3f60-470b-8e8d-adca2f88e017 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:05.940507 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0114923-3f60-470b-8e8d-adca2f88e017 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:05.941974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0114923-3f60-470b-8e8d-adca2f88e017 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:05.960009 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 376/746] 10.222.0.64 () {56 vars in 1149 bytes} [Mon May 23 15:22:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:05.960794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0114923-3f60-470b-8e8d-adca2f88e017 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:05.977383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 371/747] 10.222.0.64 () {60 vars in 1141 bytes} [Mon May 23 15:22:05 2022] POST /identity/v3/services => generated 240 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:22:05.977383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a246a5b-3d3e-4df2-a8df-b31a4d1b1156 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:05.985632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:06.073296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0114923-3f60-470b-8e8d-adca2f88e017 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:06.095991 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a246a5b-3d3e-4df2-a8df-b31a4d1b1156 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:06.123494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06085c-4d78-45cc-8750-49a6fe88167b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:06.124318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06085c-4d78-45cc-8750-49a6fe88167b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:06.124606 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06085c-4d78-45cc-8750-49a6fe88167b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:06.124959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06085c-4d78-45cc-8750-49a6fe88167b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:06.133383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a80b1-f279-4698-b5da-50e5b2efc109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:06.134070 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a80b1-f279-4698-b5da-50e5b2efc109 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:06.134355 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a80b1-f279-4698-b5da-50e5b2efc109 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:06.134717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a80b1-f279-4698-b5da-50e5b2efc109 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:06.135461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a80b1-f279-4698-b5da-50e5b2efc109 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:06.136863 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a80b1-f279-4698-b5da-50e5b2efc109 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:06.137534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06085c-4d78-45cc-8750-49a6fe88167b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:06.139024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06085c-4d78-45cc-8750-49a6fe88167b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:06.157991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-8b06085c-4d78-45cc-8750-49a6fe88167b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:06.157991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:06.157991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:06.157991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:06.157991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:06.157991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:06.157991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:06.157991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:06.157991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:06.157991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:06.157991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:06.157991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:06.157991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:06.157991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:06.157991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:06.159562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:06.160600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:06.160600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:06.160600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:06.160600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:06.160600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 377/748] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:06.160600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06085c-4d78-45cc-8750-49a6fe88167b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:06.192169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:06.214698 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 372/749] 10.222.0.64 () {60 vars in 1141 bytes} [Mon May 23 15:22:05 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:22:06.218250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a80b1-f279-4698-b5da-50e5b2efc109 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:06.273303 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06085c-4d78-45cc-8750-49a6fe88167b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:06.313871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:06.331458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb62f9-2655-4898-8042-ce425b8d16a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:06.331458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb62f9-2655-4898-8042-ce425b8d16a2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:06.331738 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb62f9-2655-4898-8042-ce425b8d16a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:06.332070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb62f9-2655-4898-8042-ce425b8d16a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:06.340506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb62f9-2655-4898-8042-ce425b8d16a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:06.342316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb62f9-2655-4898-8042-ce425b8d16a2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-3cdb62f9-2655-4898-8042-ce425b8d16a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:22:06.359296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:06.360351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:06.361635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:06.361635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:06.361635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:22:06.361635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:22:06.361635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:22:06.361635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:22:06.361635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:22:06.361635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:22:06.361635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:22:06.361635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:06.361635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 378/750] 10.222.0.64 () {56 vars in 1093 bytes} [Mon May 23 15:22:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:06.366313 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb62f9-2655-4898-8042-ce425b8d16a2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:06.374007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a80b1-f279-4698-b5da-50e5b2efc109 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:06.419654 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded1d20-7d89-4437-997d-a519f8408b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:06.420324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded1d20-7d89-4437-997d-a519f8408b0f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:06.420618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded1d20-7d89-4437-997d-a519f8408b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:06.420904 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded1d20-7d89-4437-997d-a519f8408b0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:06.480776 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:06.499662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb62f9-2655-4898-8042-ce425b8d16a2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:06.510200 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded1d20-7d89-4437-997d-a519f8408b0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:06.511960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded1d20-7d89-4437-997d-a519f8408b0f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:06.545800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591d204-84c1-4c0a-9c23-304abb22d052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:06.546474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591d204-84c1-4c0a-9c23-304abb22d052 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:06.546769 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591d204-84c1-4c0a-9c23-304abb22d052 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:06.547049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591d204-84c1-4c0a-9c23-304abb22d052 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:06.548048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591d204-84c1-4c0a-9c23-304abb22d052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:06.548361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591d204-84c1-4c0a-9c23-304abb22d052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:06.551664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591d204-84c1-4c0a-9c23-304abb22d052 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:06.570715 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 379/751] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:06.571882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591d204-84c1-4c0a-9c23-304abb22d052 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:06.582074 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:06.637229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 373/752] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:22:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:22:06.638321 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded1d20-7d89-4437-997d-a519f8408b0f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:06.741321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591d204-84c1-4c0a-9c23-304abb22d052 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:06.785738 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d6b3e-5c64-4098-91eb-80e4825078d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:06.786309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded1d20-7d89-4437-997d-a519f8408b0f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:06.786435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d6b3e-5c64-4098-91eb-80e4825078d5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:06.786686 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d6b3e-5c64-4098-91eb-80e4825078d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:06.786961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d6b3e-5c64-4098-91eb-80e4825078d5 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:06.787807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d6b3e-5c64-4098-91eb-80e4825078d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:06.788119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d6b3e-5c64-4098-91eb-80e4825078d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:06.790860 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d6b3e-5c64-4098-91eb-80e4825078d5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:06.819284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 380/753] 10.222.0.64 () {56 vars in 1133 bytes} [Mon May 23 15:22:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:06.819284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d6b3e-5c64-4098-91eb-80e4825078d5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:06.826369 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:06.840731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286508df-cc8a-4918-8a26-8d7d020ecf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:06.840731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286508df-cc8a-4918-8a26-8d7d020ecf81 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:06.841297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286508df-cc8a-4918-8a26-8d7d020ecf81 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:06.842169 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286508df-cc8a-4918-8a26-8d7d020ecf81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:06.852207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286508df-cc8a-4918-8a26-8d7d020ecf81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:06.854167 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286508df-cc8a-4918-8a26-8d7d020ecf81 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:06.872363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-286508df-cc8a-4918-8a26-8d7d020ecf81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:06.872363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:06.872363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:06.872363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:06.872363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:06.872363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:06.872363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:06.872363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:06.872363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:06.872363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:06.872363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:06.872363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:06.872363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:06.872363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:06.872363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:06.873557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:06.874682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:06.874682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:06.874682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:06.874682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:06.874682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:06.874682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:06.874682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:06.874682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:06.882803 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 374/754] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:06.882803 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286508df-cc8a-4918-8a26-8d7d020ecf81 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:06.906889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:06.935818 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d6b3e-5c64-4098-91eb-80e4825078d5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:06.974520 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7a001-3c5f-4f35-a058-2ff21fa5b8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:06.975781 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7a001-3c5f-4f35-a058-2ff21fa5b8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:06.976297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7a001-3c5f-4f35-a058-2ff21fa5b8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:06.976801 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7a001-3c5f-4f35-a058-2ff21fa5b8d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:06.988961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7a001-3c5f-4f35-a058-2ff21fa5b8d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:06.990715 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7a001-3c5f-4f35-a058-2ff21fa5b8d2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:06.991127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286508df-cc8a-4918-8a26-8d7d020ecf81 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:07.010203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-93d7a001-3c5f-4f35-a058-2ff21fa5b8d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:07.010203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:07.010203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:07.010203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:07.010203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:07.010203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:07.010203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:07.010203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:07.010203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:07.010203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:07.010203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:07.010203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:07.010203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:07.010203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:07.010203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:07.011148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:07.012061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:07.012061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:07.012061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:07.012061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:07.014479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 381/755] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:07.015576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7a001-3c5f-4f35-a058-2ff21fa5b8d2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:07.031282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d049a-b0cb-4f5a-b8dd-af8eca0cc69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:07.032220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d049a-b0cb-4f5a-b8dd-af8eca0cc69f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:07.032507 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d049a-b0cb-4f5a-b8dd-af8eca0cc69f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:07.032782 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d049a-b0cb-4f5a-b8dd-af8eca0cc69f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:07.044657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d049a-b0cb-4f5a-b8dd-af8eca0cc69f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:07.046270 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d049a-b0cb-4f5a-b8dd-af8eca0cc69f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:07.066156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-772d049a-b0cb-4f5a-b8dd-af8eca0cc69f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:07.066156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:07.066156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:07.066156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:07.066156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:07.066156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:07.066156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:07.066156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:07.066156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:07.066156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:07.066156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:07.066156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:07.066156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:07.066156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:07.066156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:07.067134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:07.068076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:07.068076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:07.068076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:07.068076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:07.072128 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 375/756] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:07.073122 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d049a-b0cb-4f5a-b8dd-af8eca0cc69f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:07.084389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:07.132613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7a001-3c5f-4f35-a058-2ff21fa5b8d2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:07.151735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:07.171790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599c403-71ad-4644-ad19-556baafe3a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:07.172884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599c403-71ad-4644-ad19-556baafe3a54 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:07.173419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599c403-71ad-4644-ad19-556baafe3a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:07.173908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599c403-71ad-4644-ad19-556baafe3a54 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:07.175030 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599c403-71ad-4644-ad19-556baafe3a54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:07.175572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599c403-71ad-4644-ad19-556baafe3a54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:07.177323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599c403-71ad-4644-ad19-556baafe3a54 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:07.183186 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d049a-b0cb-4f5a-b8dd-af8eca0cc69f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:07.195005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 382/757] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:07.195919 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599c403-71ad-4644-ad19-556baafe3a54 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:07.219415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f943d33-9901-4c28-9c41-eeb624194808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:07.220088 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f943d33-9901-4c28-9c41-eeb624194808 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:07.220375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f943d33-9901-4c28-9c41-eeb624194808 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:07.220649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f943d33-9901-4c28-9c41-eeb624194808 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:07.221897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f943d33-9901-4c28-9c41-eeb624194808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:07.222262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f943d33-9901-4c28-9c41-eeb624194808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:07.223749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f943d33-9901-4c28-9c41-eeb624194808 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:07.241253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 376/758] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:07.242267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f943d33-9901-4c28-9c41-eeb624194808 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:07.306263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599c403-71ad-4644-ad19-556baafe3a54 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:07.325349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:07.342466 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ba2ef-bf16-4fef-8f23-e267fef5ee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:07.343304 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ba2ef-bf16-4fef-8f23-e267fef5ee92 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:07.343597 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ba2ef-bf16-4fef-8f23-e267fef5ee92 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:07.344021 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ba2ef-bf16-4fef-8f23-e267fef5ee92 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:07.344779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ba2ef-bf16-4fef-8f23-e267fef5ee92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:07.345117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ba2ef-bf16-4fef-8f23-e267fef5ee92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:07.346745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ba2ef-bf16-4fef-8f23-e267fef5ee92 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:07.360553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 383/759] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:07.361281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f943d33-9901-4c28-9c41-eeb624194808 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:07.361396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ba2ef-bf16-4fef-8f23-e267fef5ee92 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:07.396258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:07.401976 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83237fe5-be96-48ed-ab5b-779c60353816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:07.402652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83237fe5-be96-48ed-ab5b-779c60353816 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:07.402943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83237fe5-be96-48ed-ab5b-779c60353816 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:07.403226 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83237fe5-be96-48ed-ab5b-779c60353816 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:07.404162 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83237fe5-be96-48ed-ab5b-779c60353816 None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=f8c79c36c7204368816593f0e473191b) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:07.404484 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83237fe5-be96-48ed-ab5b-779c60353816 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:07.405984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83237fe5-be96-48ed-ab5b-779c60353816 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:07.420279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 377/760] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:22:07 2022] GET /identity/v3/role_assignments?user.id=f8c79c36c7204368816593f0e473191b&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:07.421569 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83237fe5-be96-48ed-ab5b-779c60353816 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:07.541312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ba2ef-bf16-4fef-8f23-e267fef5ee92 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:07.598813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b756729-5628-47f5-9c04-e5d8acb6eae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:07.600047 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b756729-5628-47f5-9c04-e5d8acb6eae6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:07.602009 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b756729-5628-47f5-9c04-e5d8acb6eae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:07.602500 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b756729-5628-47f5-9c04-e5d8acb6eae6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:07.621311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b756729-5628-47f5-9c04-e5d8acb6eae6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:07.623319 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b756729-5628-47f5-9c04-e5d8acb6eae6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:07.640970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-8b756729-5628-47f5-9c04-e5d8acb6eae6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:07.640970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:07.640970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:07.640970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:07.640970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:07.640970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:07.640970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:07.640970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:07.640970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:07.640970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:07.640970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:07.640970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:07.640970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:07.640970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:07.640970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:07.643422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:07.644474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:07.644474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:07.644474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:07.644474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:07.644474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:07.644474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:07.644474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:07.644474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:07.644474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 384/761] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:07.649648 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b756729-5628-47f5-9c04-e5d8acb6eae6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:07.651279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83237fe5-be96-48ed-ab5b-779c60353816 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:07.738754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5fa796-7d75-4569-a6fc-4a30d25f8d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:07.739563 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5fa796-7d75-4569-a6fc-4a30d25f8d39 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:07.739850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5fa796-7d75-4569-a6fc-4a30d25f8d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:07.740126 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5fa796-7d75-4569-a6fc-4a30d25f8d39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:07.757717 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:07.758219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5fa796-7d75-4569-a6fc-4a30d25f8d39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:07.759382 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5fa796-7d75-4569-a6fc-4a30d25f8d39 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:07.793539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-2c5fa796-7d75-4569-a6fc-4a30d25f8d39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:07.793539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:07.793539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:07.793539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:07.793539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:07.793539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:07.793539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:07.793539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:07.793539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:07.793539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:07.793539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:07.793539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:07.793539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:07.793539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:07.793539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:07.794518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:07.795887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:07.795887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:07.795887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:07.795887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:07.795887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:07.795887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:07.795887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:07.795887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:07.795887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 378/762] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:07.800741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5fa796-7d75-4569-a6fc-4a30d25f8d39 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:07.824997 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b756729-5628-47f5-9c04-e5d8acb6eae6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:07.958387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:07.998845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14d186-b4a7-4198-8142-339fb9a04a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:07.999524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14d186-b4a7-4198-8142-339fb9a04a86 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:07.999805 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14d186-b4a7-4198-8142-339fb9a04a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:08.000183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14d186-b4a7-4198-8142-339fb9a04a86 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:08.000983 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14d186-b4a7-4198-8142-339fb9a04a86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:08.001319 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14d186-b4a7-4198-8142-339fb9a04a86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:08.003429 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14d186-b4a7-4198-8142-339fb9a04a86 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:08.029040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 385/763] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:08.032427 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5fa796-7d75-4569-a6fc-4a30d25f8d39 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:08.035657 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14d186-b4a7-4198-8142-339fb9a04a86 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:08.099145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f704e-e01b-4be5-b0b7-356b7f8d27bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:08.099825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f704e-e01b-4be5-b0b7-356b7f8d27bc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:08.100115 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f704e-e01b-4be5-b0b7-356b7f8d27bc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:08.100390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f704e-e01b-4be5-b0b7-356b7f8d27bc None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:08.101218 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f704e-e01b-4be5-b0b7-356b7f8d27bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:08.101532 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f704e-e01b-4be5-b0b7-356b7f8d27bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:08.103138 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f704e-e01b-4be5-b0b7-356b7f8d27bc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:08.129109 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 379/764] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:08.134430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f704e-e01b-4be5-b0b7-356b7f8d27bc None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:08.168059 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:08.173080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14d186-b4a7-4198-8142-339fb9a04a86 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:08.227684 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf44492-3611-45b5-9e7c-56d01fa65013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:08.228269 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf44492-3611-45b5-9e7c-56d01fa65013 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:08.228552 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf44492-3611-45b5-9e7c-56d01fa65013 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:08.228824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf44492-3611-45b5-9e7c-56d01fa65013 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:08.229684 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf44492-3611-45b5-9e7c-56d01fa65013 None admin] RBAC: Adding query filter params (role.id=3d6e986dd2264e3cb7e01e25dbac6f97, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=08dad0888c8542f4aed748a1a35db2c4) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:08.229996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf44492-3611-45b5-9e7c-56d01fa65013 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:08.231728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf44492-3611-45b5-9e7c-56d01fa65013 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:08.245346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 386/765] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:22:08 2022] GET /identity/v3/role_assignments?user.id=08dad0888c8542f4aed748a1a35db2c4&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=3d6e986dd2264e3cb7e01e25dbac6f97 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:08.263352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f704e-e01b-4be5-b0b7-356b7f8d27bc None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:08.281180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:08.302252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb218d-11fa-4d00-90bf-2ca361bd058d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:08.303166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb218d-11fa-4d00-90bf-2ca361bd058d None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:08.303453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb218d-11fa-4d00-90bf-2ca361bd058d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:08.303731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb218d-11fa-4d00-90bf-2ca361bd058d None admin] PATH_INFO: `/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/8da96bed73794fee8f719ec904936164/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:08.305926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb218d-11fa-4d00-90bf-2ca361bd058d None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d81eafac474f37a71ee5cc292d172f, user_id=8da96bed73794fee8f719ec904936164, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:08.307393 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb218d-11fa-4d00-90bf-2ca361bd058d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:08.375235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 380/766] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:22:08 2022] PUT /identity/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/8da96bed73794fee8f719ec904936164/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:22:08.425759 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbe125-83c7-4087-b9dd-2295563dd3b2 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:08.426093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbe125-83c7-4087-b9dd-2295563dd3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:08.426328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbe125-83c7-4087-b9dd-2295563dd3b2 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:08.428058 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 387/767] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:08.434226 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826e37c-44ee-4ad6-9456-6a5a0a8d9b6c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:08.434543 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826e37c-44ee-4ad6-9456-6a5a0a8d9b6c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:08.434810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826e37c-44ee-4ad6-9456-6a5a0a8d9b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:08.458384 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2826e37c-44ee-4ad6-9456-6a5a0a8d9b6c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:08.459664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2826e37c-44ee-4ad6-9456-6a5a0a8d9b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:08.547110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 381/768] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:08.583852 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbe125-83c7-4087-b9dd-2295563dd3b2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:08.744227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbe125-83c7-4087-b9dd-2295563dd3b2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:08.761505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e3bb2-d014-47fe-8753-07d6216d74fd None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:08.762044 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e3bb2-d014-47fe-8753-07d6216d74fd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:08.762044 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e3bb2-d014-47fe-8753-07d6216d74fd None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:08.763642 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 382/769] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:08.769180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b5a73-fde0-4d71-a74e-73b023b1045a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:08.769471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b5a73-fde0-4d71-a74e-73b023b1045a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:08.769758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b5a73-fde0-4d71-a74e-73b023b1045a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:08.801328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0c5b5a73-fde0-4d71-a74e-73b023b1045a None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:08.802737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0c5b5a73-fde0-4d71-a74e-73b023b1045a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:08.816396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206cbfc-d25c-4d97-a328-2f089977261a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:08.817291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206cbfc-d25c-4d97-a328-2f089977261a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:08.817580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206cbfc-d25c-4d97-a328-2f089977261a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:08.818096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206cbfc-d25c-4d97-a328-2f089977261a None admin] PATH_INFO: `/v3/services/network` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:08.820454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206cbfc-d25c-4d97-a328-2f089977261a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:08.822280 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206cbfc-d25c-4d97-a328-2f089977261a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d206cbfc-d25c-4d97-a328-2f089977261a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:08.846254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:08.847333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:08.848223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:08.848223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:08.848223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:08.848223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:08.848223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:08.848223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:08.848223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:08.848223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:08.848223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:08.848223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:08.848223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:08.848223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:08.848223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:22:08.848223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:08.848223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 388/770] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:08 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:08.854572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206cbfc-d25c-4d97-a328-2f089977261a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:08.888058 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:08.943924 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 383/771] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:08.965685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206cbfc-d25c-4d97-a328-2f089977261a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:08.978103 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b5a73-fde0-4d71-a74e-73b023b1045a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:09.008649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb93f5-d1ee-407d-937a-5966fb0b7407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:09.009356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb93f5-d1ee-407d-937a-5966fb0b7407 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:09.009646 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb93f5-d1ee-407d-937a-5966fb0b7407 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:09.009933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb93f5-d1ee-407d-937a-5966fb0b7407 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:09.010647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb93f5-d1ee-407d-937a-5966fb0b7407 None admin] RBAC: Adding query filter params (name=network) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:09.010961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb93f5-d1ee-407d-937a-5966fb0b7407 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:09.012336 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb93f5-d1ee-407d-937a-5966fb0b7407 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:09.021786 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:09.025511 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 389/772] 10.222.0.64 () {56 vars in 1112 bytes} [Mon May 23 15:22:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:09.030741 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb93f5-d1ee-407d-937a-5966fb0b7407 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:09.126382 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:09.131418 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b5a73-fde0-4d71-a74e-73b023b1045a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:09.136843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb93f5-d1ee-407d-937a-5966fb0b7407 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:09.180191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e55a12-0f87-4be7-9c66-49708aebd76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:09.180871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e55a12-0f87-4be7-9c66-49708aebd76f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:09.181187 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e55a12-0f87-4be7-9c66-49708aebd76f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:09.181474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e55a12-0f87-4be7-9c66-49708aebd76f None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:09.182190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e55a12-0f87-4be7-9c66-49708aebd76f None admin] RBAC: Adding query filter params (type=network) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:09.182504 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e55a12-0f87-4be7-9c66-49708aebd76f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:09.183881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e55a12-0f87-4be7-9c66-49708aebd76f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:09.194528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 390/773] 10.222.0.64 () {56 vars in 1112 bytes} [Mon May 23 15:22:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:09.200579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e55a12-0f87-4be7-9c66-49708aebd76f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:09.202485 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fdfe3-f17e-4231-a7e0-9010abc3baeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:09.203306 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fdfe3-f17e-4231-a7e0-9010abc3baeb None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:09.203587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fdfe3-f17e-4231-a7e0-9010abc3baeb None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:09.203860 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fdfe3-f17e-4231-a7e0-9010abc3baeb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:09.204526 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10fdfe3-f17e-4231-a7e0-9010abc3baeb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:09.206154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10fdfe3-f17e-4231-a7e0-9010abc3baeb None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-f10fdfe3-f17e-4231-a7e0-9010abc3baeb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:09.230243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:09.232423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:09.234469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:09.234469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:09.234469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:09.234469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:09.234469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:09.234469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:09.234469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:09.234469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:09.234469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:09.234469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:09.234469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:09.234469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:09.234469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:09.234469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:22:09.234469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:09.242972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 384/774] 10.222.0.64 () {56 vars in 1129 bytes} [Mon May 23 15:22:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:09.243880 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fdfe3-f17e-4231-a7e0-9010abc3baeb None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:09.394589 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e55a12-0f87-4be7-9c66-49708aebd76f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:09.397748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:09.400824 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fdfe3-f17e-4231-a7e0-9010abc3baeb None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:09.465686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f2003-17ad-453b-ac19-15b9515df917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:09.466360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f2003-17ad-453b-ac19-15b9515df917 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:09.466652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f2003-17ad-453b-ac19-15b9515df917 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:09.466938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f2003-17ad-453b-ac19-15b9515df917 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:09.467662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f2003-17ad-453b-ac19-15b9515df917 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:09.467976 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f2003-17ad-453b-ac19-15b9515df917 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:09.469400 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f2003-17ad-453b-ac19-15b9515df917 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:09.471639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf215e-556b-4e6c-94d4-15a94f39fce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:09.472404 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf215e-556b-4e6c-94d4-15a94f39fce8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:09.472692 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf215e-556b-4e6c-94d4-15a94f39fce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:09.472971 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf215e-556b-4e6c-94d4-15a94f39fce8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:09.473832 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf215e-556b-4e6c-94d4-15a94f39fce8 None admin] RBAC: Adding query filter params (interface=public, service_id=91bdd446b8af4cc6b80eacb15f1cfe75, region_id=RegionOne) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:09.474149 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf215e-556b-4e6c-94d4-15a94f39fce8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:09.475512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf215e-556b-4e6c-94d4-15a94f39fce8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:09.484152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 385/775] 10.222.0.64 () {56 vars in 1124 bytes} [Mon May 23 15:22:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:09.488209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 391/776] 10.222.0.64 () {56 vars in 1251 bytes} [Mon May 23 15:22:09 2022] GET /identity/v3/endpoints?service_id=91bdd446b8af4cc6b80eacb15f1cfe75&interface=public®ion_id=RegionOne => generated 194 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:09.493769 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf215e-556b-4e6c-94d4-15a94f39fce8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:09.493987 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f2003-17ad-453b-ac19-15b9515df917 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:09.530670 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:09.600851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f2003-17ad-453b-ac19-15b9515df917 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:09.647619 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefafbb-6fdb-4d78-9f27-67e6e9206726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:09.648264 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefafbb-6fdb-4d78-9f27-67e6e9206726 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:09.648555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefafbb-6fdb-4d78-9f27-67e6e9206726 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:09.648837 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefafbb-6fdb-4d78-9f27-67e6e9206726 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:09.649555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fefafbb-6fdb-4d78-9f27-67e6e9206726 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:09.650979 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fefafbb-6fdb-4d78-9f27-67e6e9206726 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:09.661454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 386/777] 10.222.0.64 () {56 vars in 1087 bytes} [Mon May 23 15:22:09 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:09.678149 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf215e-556b-4e6c-94d4-15a94f39fce8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:09.746441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40df87-41cd-44c3-92a6-6d8768ea9c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:09.747496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40df87-41cd-44c3-92a6-6d8768ea9c3f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:09.747971 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40df87-41cd-44c3-92a6-6d8768ea9c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:09.748447 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40df87-41cd-44c3-92a6-6d8768ea9c3f None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:09.754035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40df87-41cd-44c3-92a6-6d8768ea9c3f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:09.754622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40df87-41cd-44c3-92a6-6d8768ea9c3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:09.756235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40df87-41cd-44c3-92a6-6d8768ea9c3f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:09.762604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 392/778] 10.222.0.64 () {56 vars in 1124 bytes} [Mon May 23 15:22:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:09.774241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefafbb-6fdb-4d78-9f27-67e6e9206726 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:09.825486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:09.920275 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefafbb-6fdb-4d78-9f27-67e6e9206726 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:09.963558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e13ce4-4641-40f1-8153-8d7f90c1ab16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:09.964257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e13ce4-4641-40f1-8153-8d7f90c1ab16 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:09.964543 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e13ce4-4641-40f1-8153-8d7f90c1ab16 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:09.964823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e13ce4-4641-40f1-8153-8d7f90c1ab16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:09.965680 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e13ce4-4641-40f1-8153-8d7f90c1ab16 None admin] RBAC: Adding query filter params (interface=public, service_id=58ccc77230864215a14e728aae07bb5c, region_id=RegionOne) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:09.965996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e13ce4-4641-40f1-8153-8d7f90c1ab16 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:09.967423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e13ce4-4641-40f1-8153-8d7f90c1ab16 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:09.977053 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 387/779] 10.222.0.64 () {56 vars in 1251 bytes} [Mon May 23 15:22:09 2022] GET /identity/v3/endpoints?service_id=58ccc77230864215a14e728aae07bb5c&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:09.982591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40df87-41cd-44c3-92a6-6d8768ea9c3f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:10.085485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:10.105366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40df87-41cd-44c3-92a6-6d8768ea9c3f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:10.147374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57f931-a3ab-4951-98ed-b13a6cbdc6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:10.148053 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57f931-a3ab-4951-98ed-b13a6cbdc6da None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:10.148342 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57f931-a3ab-4951-98ed-b13a6cbdc6da None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:10.148625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57f931-a3ab-4951-98ed-b13a6cbdc6da None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:10.149357 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57f931-a3ab-4951-98ed-b13a6cbdc6da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:10.150725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57f931-a3ab-4951-98ed-b13a6cbdc6da None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:10.156368 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 393/780] 10.222.0.64 () {56 vars in 1087 bytes} [Mon May 23 15:22:09 2022] GET /identity/v3/services => generated 772 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:10.162177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b21d8a-29d4-4b2f-866a-bdd69a5ae7b2 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:10.162765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b21d8a-29d4-4b2f-866a-bdd69a5ae7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:10.163238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b21d8a-29d4-4b2f-866a-bdd69a5ae7b2 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:10.169305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 388/781] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:10.171105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18581cf7-cfcb-4327-8222-af9813016e90 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:10.171347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18581cf7-cfcb-4327-8222-af9813016e90 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:10.171633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18581cf7-cfcb-4327-8222-af9813016e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:10.191532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-18581cf7-cfcb-4327-8222-af9813016e90 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:10.192794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-18581cf7-cfcb-4327-8222-af9813016e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:10.266852 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 394/782] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:10.306063 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b21d8a-29d4-4b2f-866a-bdd69a5ae7b2 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:10.438470 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:10.453902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b21d8a-29d4-4b2f-866a-bdd69a5ae7b2 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:10.504833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580dd0a4-cbb0-4d28-808f-c60aef71293b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:10.505910 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dd0a4-cbb0-4d28-808f-c60aef71293b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:10.506369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dd0a4-cbb0-4d28-808f-c60aef71293b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:10.506792 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dd0a4-cbb0-4d28-808f-c60aef71293b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:10.520569 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580dd0a4-cbb0-4d28-808f-c60aef71293b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-580dd0a4-cbb0-4d28-808f-c60aef71293b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:10.528548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:10.529707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:10.530685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:10.530685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:10.530685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:10.530685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:10.530685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:10.530685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:10.530685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:10.530685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:10.530685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:10.530685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:10.530685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:10.530685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:10.535268 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 389/783] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:22:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:10.537969 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18581cf7-cfcb-4327-8222-af9813016e90 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:10.618882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:10.643299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18581cf7-cfcb-4327-8222-af9813016e90 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:10.697376 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73459d-f54d-421e-9f06-bc1b5d38a5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:10.697994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73459d-f54d-421e-9f06-bc1b5d38a5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:10.698306 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73459d-f54d-421e-9f06-bc1b5d38a5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:10.698589 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73459d-f54d-421e-9f06-bc1b5d38a5d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:10.699340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73459d-f54d-421e-9f06-bc1b5d38a5d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:10.699650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73459d-f54d-421e-9f06-bc1b5d38a5d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:10.701018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73459d-f54d-421e-9f06-bc1b5d38a5d2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:10.710147 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 395/784] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:22:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:10.716073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580dd0a4-cbb0-4d28-808f-c60aef71293b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:10.871609 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580dd0a4-cbb0-4d28-808f-c60aef71293b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:10.922678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39603c8-62e9-4aee-a8f6-85379f57b0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:10.923493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39603c8-62e9-4aee-a8f6-85379f57b0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:10.923784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39603c8-62e9-4aee-a8f6-85379f57b0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:10.924129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39603c8-62e9-4aee-a8f6-85379f57b0b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:10.938215 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39603c8-62e9-4aee-a8f6-85379f57b0b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:10.940239 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39603c8-62e9-4aee-a8f6-85379f57b0b2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:10.969505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c39603c8-62e9-4aee-a8f6-85379f57b0b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:10.969505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:10.969505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:10.969505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:10.969505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:10.969505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:10.969505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:10.969505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:10.969505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:10.969505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:10.969505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:10.969505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:10.969505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:10.969505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:10.969505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:10.970449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:10.971935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:10.971935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:10.971935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:10.971935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:10.971935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 390/785] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:10.977399 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73459d-f54d-421e-9f06-bc1b5d38a5d2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:11.069651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c986e-08fd-4ea5-b5d0-23b9ce1d7023 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.069939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c986e-08fd-4ea5-b5d0-23b9ce1d7023 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.070439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c986e-08fd-4ea5-b5d0-23b9ce1d7023 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.072168 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 391/786] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:11.077421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563300e5-cde0-45b2-9a0e-17659cd2f256 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.077905 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563300e5-cde0-45b2-9a0e-17659cd2f256 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.078334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563300e5-cde0-45b2-9a0e-17659cd2f256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.099137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:11.100191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73459d-f54d-421e-9f06-bc1b5d38a5d2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:11.113912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-563300e5-cde0-45b2-9a0e-17659cd2f256 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:11.115920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-563300e5-cde0-45b2-9a0e-17659cd2f256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:11.137188 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ebcfc-b323-4ac8-9e5c-2af48fff98e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:11.137958 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ebcfc-b323-4ac8-9e5c-2af48fff98e5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.138319 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ebcfc-b323-4ac8-9e5c-2af48fff98e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.138671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ebcfc-b323-4ac8-9e5c-2af48fff98e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.139640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ebcfc-b323-4ac8-9e5c-2af48fff98e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:11.140042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ebcfc-b323-4ac8-9e5c-2af48fff98e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:11.141849 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ebcfc-b323-4ac8-9e5c-2af48fff98e5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:11.151660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:11.158701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 396/787] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:11.160356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22296b7d-6c99-4109-b70f-c0c1404c0652 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.164612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22296b7d-6c99-4109-b70f-c0c1404c0652 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.164612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22296b7d-6c99-4109-b70f-c0c1404c0652 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.164612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 397/788] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:11.168516 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22296b7d-6c99-4109-b70f-c0c1404c0652 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:11.208078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 392/789] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:11.209758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb835ed-a172-43ba-a2aa-c9b4191a517d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.210058 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb835ed-a172-43ba-a2aa-c9b4191a517d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.210340 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb835ed-a172-43ba-a2aa-c9b4191a517d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.231070 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-bcb835ed-a172-43ba-a2aa-c9b4191a517d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:11.232725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bcb835ed-a172-43ba-a2aa-c9b4191a517d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:11.286671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22296b7d-6c99-4109-b70f-c0c1404c0652 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:11.327099 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c2d27-2d4a-4791-94f5-d6ec0c3a8653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:11.328277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c2d27-2d4a-4791-94f5-d6ec0c3a8653 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.328763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c2d27-2d4a-4791-94f5-d6ec0c3a8653 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.329030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 393/790] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:11.329512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c2d27-2d4a-4791-94f5-d6ec0c3a8653 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.333623 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb835ed-a172-43ba-a2aa-c9b4191a517d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:11.347590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c2d27-2d4a-4791-94f5-d6ec0c3a8653 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:11.349336 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c2d27-2d4a-4791-94f5-d6ec0c3a8653 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-c19c2d27-2d4a-4791-94f5-d6ec0c3a8653 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:22:11.367012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:11.368019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:11.368995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:11.368995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:11.368995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:22:11.368995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:22:11.368995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:22:11.368995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:22:11.368995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:22:11.368995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:22:11.368995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:22:11.368995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:11.371606 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 398/791] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:22:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:11.372602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c2d27-2d4a-4791-94f5-d6ec0c3a8653 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:11.427180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:11.438160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:11.479515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb835ed-a172-43ba-a2aa-c9b4191a517d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:11.485201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c2d27-2d4a-4791-94f5-d6ec0c3a8653 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:11.520722 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705379a-3fbe-4c73-b276-3502a3d5267a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:11.521981 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705379a-3fbe-4c73-b276-3502a3d5267a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.522470 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705379a-3fbe-4c73-b276-3502a3d5267a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.522949 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705379a-3fbe-4c73-b276-3502a3d5267a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.533100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489df72-881b-4d91-8502-9b0da94e7f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:11.534111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489df72-881b-4d91-8502-9b0da94e7f49 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.534401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489df72-881b-4d91-8502-9b0da94e7f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.534729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489df72-881b-4d91-8502-9b0da94e7f49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.536011 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705379a-3fbe-4c73-b276-3502a3d5267a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-3705379a-3fbe-4c73-b276-3502a3d5267a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:11.544424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:11.545385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:11.546254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:11.546254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:11.546254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:11.546254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:11.546254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:11.546254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:11.546254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:11.546254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:11.546254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:11.546254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:11.546254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:11.546254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:11.546896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c489df72-881b-4d91-8502-9b0da94e7f49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:11.549001 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 394/792] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:22:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:11.550115 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705379a-3fbe-4c73-b276-3502a3d5267a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-c489df72-881b-4d91-8502-9b0da94e7f49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:11.555802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:11.556733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:11.557927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:11.557927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:11.557927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:11.557927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:11.557927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:11.557927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:11.557927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:11.557927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:11.557927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:11.557927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:11.557927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:22:11.557927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:11.561819 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 399/793] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:22:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:11.562653 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489df72-881b-4d91-8502-9b0da94e7f49 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:11.672873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705379a-3fbe-4c73-b276-3502a3d5267a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:11.697256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489df72-881b-4d91-8502-9b0da94e7f49 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:11.700303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:11.711732 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf187f7-9024-4473-b637-1ff5bcb9895d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:11.712535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf187f7-9024-4473-b637-1ff5bcb9895d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.713765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf187f7-9024-4473-b637-1ff5bcb9895d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.713765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf187f7-9024-4473-b637-1ff5bcb9895d None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.714292 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf187f7-9024-4473-b637-1ff5bcb9895d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:11.714601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf187f7-9024-4473-b637-1ff5bcb9895d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:11.716180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf187f7-9024-4473-b637-1ff5bcb9895d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:11.721870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:11.742468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 395/794] 10.222.0.64 () {56 vars in 1135 bytes} [Mon May 23 15:22:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:11.742468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf187f7-9024-4473-b637-1ff5bcb9895d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:11.748632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394f1c9-5fca-47d5-83ae-48c4b6d66863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:11.749399 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394f1c9-5fca-47d5-83ae-48c4b6d66863 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.749690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394f1c9-5fca-47d5-83ae-48c4b6d66863 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.750022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394f1c9-5fca-47d5-83ae-48c4b6d66863 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.750839 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394f1c9-5fca-47d5-83ae-48c4b6d66863 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:11.751158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394f1c9-5fca-47d5-83ae-48c4b6d66863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:11.754273 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394f1c9-5fca-47d5-83ae-48c4b6d66863 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:11.769107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 400/795] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:22:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:11.769107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394f1c9-5fca-47d5-83ae-48c4b6d66863 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:11.870213 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf187f7-9024-4473-b637-1ff5bcb9895d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:11.896343 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394f1c9-5fca-47d5-83ae-48c4b6d66863 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:11.907270 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075808b-b187-4ed8-acc1-14bd68a413b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:11.907952 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075808b-b187-4ed8-acc1-14bd68a413b1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.908268 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075808b-b187-4ed8-acc1-14bd68a413b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.909227 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075808b-b187-4ed8-acc1-14bd68a413b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.910042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075808b-b187-4ed8-acc1-14bd68a413b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:11.910402 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075808b-b187-4ed8-acc1-14bd68a413b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:11.911786 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075808b-b187-4ed8-acc1-14bd68a413b1 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:11.920583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 396/796] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:22:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:11.921429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075808b-b187-4ed8-acc1-14bd68a413b1 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:11.933150 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2269e3-f3f2-46d5-99d4-f3238738f140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:11.933957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2269e3-f3f2-46d5-99d4-f3238738f140 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:11.934249 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2269e3-f3f2-46d5-99d4-f3238738f140 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:11.934531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2269e3-f3f2-46d5-99d4-f3238738f140 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:11.945813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2269e3-f3f2-46d5-99d4-f3238738f140 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:11.947280 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2269e3-f3f2-46d5-99d4-f3238738f140 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:11.965563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-bf2269e3-f3f2-46d5-99d4-f3238738f140 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:11.965563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:11.965563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:11.965563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:11.965563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:11.965563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:11.965563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:11.965563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:11.965563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:11.965563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:11.965563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:11.965563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:11.965563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:11.965563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:11.965563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:11.966515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:11.967407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:11.967407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:11.967407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:11.967407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:11.967407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 401/797] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:11.968106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2269e3-f3f2-46d5-99d4-f3238738f140 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:11.977352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:12.001665 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:12.040861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075808b-b187-4ed8-acc1-14bd68a413b1 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:12.079832 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2269e3-f3f2-46d5-99d4-f3238738f140 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:12.081489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6445f9-2d4f-4239-a2cc-0a43a7d5a604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:12.082299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6445f9-2d4f-4239-a2cc-0a43a7d5a604 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:12.082591 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6445f9-2d4f-4239-a2cc-0a43a7d5a604 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:12.082872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6445f9-2d4f-4239-a2cc-0a43a7d5a604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:12.094291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6445f9-2d4f-4239-a2cc-0a43a7d5a604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:12.095791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6445f9-2d4f-4239-a2cc-0a43a7d5a604 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:12.115219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-1a6445f9-2d4f-4239-a2cc-0a43a7d5a604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:12.115219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:12.115219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:12.115219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:12.115219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:12.115219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:12.115219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:12.115219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:12.115219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:12.115219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:12.115219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:12.115219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:12.115219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:12.115219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:12.115219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:12.117465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:12.119748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:12.119748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:12.119748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:12.119748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:12.119748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 397/798] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:12.119748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6445f9-2d4f-4239-a2cc-0a43a7d5a604 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:12.125093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c668d7-6c3d-489b-afda-bd4e8c2d1c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:12.125093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c668d7-6c3d-489b-afda-bd4e8c2d1c14 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:12.125093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c668d7-6c3d-489b-afda-bd4e8c2d1c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:12.125093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c668d7-6c3d-489b-afda-bd4e8c2d1c14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:12.136898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c668d7-6c3d-489b-afda-bd4e8c2d1c14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:12.138594 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c668d7-6c3d-489b-afda-bd4e8c2d1c14 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:12.157518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-16c668d7-6c3d-489b-afda-bd4e8c2d1c14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:12.157518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:12.157518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:12.157518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:12.157518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:12.157518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:12.157518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:12.157518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:12.157518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:12.157518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:12.157518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:12.157518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:12.157518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:12.157518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:12.157518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:12.159277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:12.160322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:12.160322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:12.160322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:12.160322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:12.162241 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 402/799] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:12.163281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c668d7-6c3d-489b-afda-bd4e8c2d1c14 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:12.222724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:12.228911 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6445f9-2d4f-4239-a2cc-0a43a7d5a604 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:12.265886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ea866-d3b8-4975-a72b-467ac6798dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:12.266368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ea866-d3b8-4975-a72b-467ac6798dbf None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:12.266659 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ea866-d3b8-4975-a72b-467ac6798dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:12.266942 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ea866-d3b8-4975-a72b-467ac6798dbf None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:12.267858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ea866-d3b8-4975-a72b-467ac6798dbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:12.268752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ea866-d3b8-4975-a72b-467ac6798dbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:12.270240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ea866-d3b8-4975-a72b-467ac6798dbf None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:12.270457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c668d7-6c3d-489b-afda-bd4e8c2d1c14 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:12.289260 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 398/800] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:12.290311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ea866-d3b8-4975-a72b-467ac6798dbf None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:12.310252 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8389f4b-36d3-4c75-a4d8-2a39b3c71169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:12.311206 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8389f4b-36d3-4c75-a4d8-2a39b3c71169 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:12.311656 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8389f4b-36d3-4c75-a4d8-2a39b3c71169 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:12.312095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8389f4b-36d3-4c75-a4d8-2a39b3c71169 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:12.312982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8389f4b-36d3-4c75-a4d8-2a39b3c71169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:12.313522 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8389f4b-36d3-4c75-a4d8-2a39b3c71169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:12.315877 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8389f4b-36d3-4c75-a4d8-2a39b3c71169 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:12.333459 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 403/801] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:12.334695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8389f4b-36d3-4c75-a4d8-2a39b3c71169 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:12.345825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:12.399581 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ea866-d3b8-4975-a72b-467ac6798dbf None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:12.436209 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b3fbc-8936-4868-a195-fb363d1e24e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:12.436891 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b3fbc-8936-4868-a195-fb363d1e24e6 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:12.437236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b3fbc-8936-4868-a195-fb363d1e24e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:12.437523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b3fbc-8936-4868-a195-fb363d1e24e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:12.438271 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b3fbc-8936-4868-a195-fb363d1e24e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:12.438567 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b3fbc-8936-4868-a195-fb363d1e24e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:12.441069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b3fbc-8936-4868-a195-fb363d1e24e6 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:12.445326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8389f4b-36d3-4c75-a4d8-2a39b3c71169 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:12.453976 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 399/802] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:12.454826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b3fbc-8936-4868-a195-fb363d1e24e6 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:12.475609 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:12.485165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff336a27-ade3-4c6c-9306-98251e98fa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:12.485972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff336a27-ade3-4c6c-9306-98251e98fa70 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:12.486730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff336a27-ade3-4c6c-9306-98251e98fa70 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:12.486730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff336a27-ade3-4c6c-9306-98251e98fa70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:12.493846 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff336a27-ade3-4c6c-9306-98251e98fa70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:12.495294 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff336a27-ade3-4c6c-9306-98251e98fa70 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:12.512905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-ff336a27-ade3-4c6c-9306-98251e98fa70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:12.512905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:12.512905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:12.512905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:12.512905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:12.512905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:12.512905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:12.512905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:12.512905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:12.512905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:12.512905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:12.512905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:12.512905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:12.512905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:12.512905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:12.513938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:12.514866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:12.514866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:12.514866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:12.514866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:12.514866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:12.514866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:12.514866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:12.514866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:12.514866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 404/803] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:12.515394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff336a27-ade3-4c6c-9306-98251e98fa70 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:12.519827 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:12.569565 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b3fbc-8936-4868-a195-fb363d1e24e6 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:12.608108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e52ca0-e215-40a8-8380-1334c7df8213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:12.608932 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e52ca0-e215-40a8-8380-1334c7df8213 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:12.609251 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e52ca0-e215-40a8-8380-1334c7df8213 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:12.609534 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e52ca0-e215-40a8-8380-1334c7df8213 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:12.617984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e52ca0-e215-40a8-8380-1334c7df8213 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:12.619477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e52ca0-e215-40a8-8380-1334c7df8213 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:12.626205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff336a27-ade3-4c6c-9306-98251e98fa70 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c9e52ca0-e215-40a8-8380-1334c7df8213 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:22:12.634599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:12.636368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:12.637426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:12.637426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:12.637426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:22:12.637426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:22:12.637426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:22:12.637426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:22:12.637426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:22:12.637426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:22:12.637426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:22:12.637426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:12.640641 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 400/804] 10.222.0.64 () {56 vars in 1093 bytes} [Mon May 23 15:22:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:12.642024 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272e0a6-01f8-4823-8075-eca1a83a4138 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:12.642334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272e0a6-01f8-4823-8075-eca1a83a4138 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:12.643297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272e0a6-01f8-4823-8075-eca1a83a4138 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:12.644773 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 401/805] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:12.645523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272e0a6-01f8-4823-8075-eca1a83a4138 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:12.671512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfb521-54d5-4a45-ba5b-26ff343b5619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:12.672332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfb521-54d5-4a45-ba5b-26ff343b5619 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:12.672623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfb521-54d5-4a45-ba5b-26ff343b5619 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:12.672909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfb521-54d5-4a45-ba5b-26ff343b5619 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:12.681872 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfb521-54d5-4a45-ba5b-26ff343b5619 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:12.683316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfb521-54d5-4a45-ba5b-26ff343b5619 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-e0bfb521-54d5-4a45-ba5b-26ff343b5619 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:22:12.698845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:12.699829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:12.700869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:12.700869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:12.700869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:22:12.700869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:22:12.700869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:22:12.700869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:22:12.700869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:22:12.700869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:22:12.700869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:22:12.700869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:12.704825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 405/806] 10.222.0.64 () {56 vars in 1117 bytes} [Mon May 23 15:22:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:12.705933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfb521-54d5-4a45-ba5b-26ff343b5619 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:12.739792 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:12.764934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272e0a6-01f8-4823-8075-eca1a83a4138 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:12.773682 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:12.800103 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec0451-c6d9-411a-93d7-8de252a7e92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:12.800774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec0451-c6d9-411a-93d7-8de252a7e92b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:12.801083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec0451-c6d9-411a-93d7-8de252a7e92b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:12.801380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec0451-c6d9-411a-93d7-8de252a7e92b None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:12.802427 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec0451-c6d9-411a-93d7-8de252a7e92b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:12.802742 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec0451-c6d9-411a-93d7-8de252a7e92b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:12.804191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec0451-c6d9-411a-93d7-8de252a7e92b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:12.811426 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfb521-54d5-4a45-ba5b-26ff343b5619 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:12.816363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 402/807] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:12.817686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d73af-e8a8-4458-826e-d82a6e96d025 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:12.817988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d73af-e8a8-4458-826e-d82a6e96d025 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:12.818280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d73af-e8a8-4458-826e-d82a6e96d025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:12.834188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d53d73af-e8a8-4458-826e-d82a6e96d025 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:12.835388 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d53d73af-e8a8-4458-826e-d82a6e96d025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:12.847754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef37995-7178-4abc-a50e-19ce46bb4063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:12.848424 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef37995-7178-4abc-a50e-19ce46bb4063 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:12.848710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef37995-7178-4abc-a50e-19ce46bb4063 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:12.848994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef37995-7178-4abc-a50e-19ce46bb4063 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:12.849863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef37995-7178-4abc-a50e-19ce46bb4063 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:12.850181 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef37995-7178-4abc-a50e-19ce46bb4063 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:12.851626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef37995-7178-4abc-a50e-19ce46bb4063 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:12.869211 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 406/808] 10.222.0.64 () {56 vars in 1133 bytes} [Mon May 23 15:22:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:12.870010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef37995-7178-4abc-a50e-19ce46bb4063 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:12.900790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 403/809] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:12.902158 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e716272-03f8-40c9-979e-7d6b06d1feb6 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:12.902444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e716272-03f8-40c9-979e-7d6b06d1feb6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:12.902724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e716272-03f8-40c9-979e-7d6b06d1feb6 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:12.903965 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 404/810] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:12.904638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e716272-03f8-40c9-979e-7d6b06d1feb6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:12.972066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:12.986207 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef37995-7178-4abc-a50e-19ce46bb4063 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:13.006677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e716272-03f8-40c9-979e-7d6b06d1feb6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:13.008863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:13.020139 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab945612-316c-4b25-a24e-41ceddaca044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:13.020814 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab945612-316c-4b25-a24e-41ceddaca044 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:13.021126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab945612-316c-4b25-a24e-41ceddaca044 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:13.021411 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab945612-316c-4b25-a24e-41ceddaca044 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:13.022255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab945612-316c-4b25-a24e-41ceddaca044 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:13.022574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab945612-316c-4b25-a24e-41ceddaca044 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:13.024091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab945612-316c-4b25-a24e-41ceddaca044 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:13.040468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6eedeb-e651-42f7-addb-53280340dd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:13.041389 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6eedeb-e651-42f7-addb-53280340dd08 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:13.041625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 407/811] 10.222.0.64 () {56 vars in 1149 bytes} [Mon May 23 15:22:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:13.041749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6eedeb-e651-42f7-addb-53280340dd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:13.041947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6eedeb-e651-42f7-addb-53280340dd08 None admin] PATH_INFO: `/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/f8c79c36c7204368816593f0e473191b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:13.042449 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab945612-316c-4b25-a24e-41ceddaca044 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:13.043917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6eedeb-e651-42f7-addb-53280340dd08 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d81eafac474f37a71ee5cc292d172f, user_id=f8c79c36c7204368816593f0e473191b, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:13.045394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6eedeb-e651-42f7-addb-53280340dd08 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:13.104914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 405/812] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:22:12 2022] PUT /identity/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/f8c79c36c7204368816593f0e473191b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:22:13.106222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b3cc8-2517-41a3-b129-ed154480d674 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:13.106510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b3cc8-2517-41a3-b129-ed154480d674 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:13.106790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b3cc8-2517-41a3-b129-ed154480d674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:13.137039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-818b3cc8-2517-41a3-b129-ed154480d674 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:13.139204 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-818b3cc8-2517-41a3-b129-ed154480d674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:13.151160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab945612-316c-4b25-a24e-41ceddaca044 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:13.187632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05b773-3bf0-486e-a25b-52fd2bfb6199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:13.189223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05b773-3bf0-486e-a25b-52fd2bfb6199 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:13.190432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05b773-3bf0-486e-a25b-52fd2bfb6199 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:13.190902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05b773-3bf0-486e-a25b-52fd2bfb6199 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:13.229120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 406/813] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:13.229120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b3cc8-2517-41a3-b129-ed154480d674 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:13.259590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:13.280629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd05b773-3bf0-486e-a25b-52fd2bfb6199 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:13.282181 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd05b773-3bf0-486e-a25b-52fd2bfb6199 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:13.295863 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:13.347100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b3cc8-2517-41a3-b129-ed154480d674 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:13.382507 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbe3bf-e1c2-4e37-b50e-c03711808f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:13.383311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbe3bf-e1c2-4e37-b50e-c03711808f1f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:13.383600 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbe3bf-e1c2-4e37-b50e-c03711808f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:13.383874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbe3bf-e1c2-4e37-b50e-c03711808f1f None admin] PATH_INFO: `/v3/services/network` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:13.384524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbe3bf-e1c2-4e37-b50e-c03711808f1f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:13.385954 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbe3bf-e1c2-4e37-b50e-c03711808f1f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:13.395545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 408/814] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:22:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:22:13.396591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05b773-3bf0-486e-a25b-52fd2bfb6199 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-1dbbe3bf-e1c2-4e37-b50e-c03711808f1f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:13.397909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:13.398935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:13.399813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:13.399813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:13.399813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:13.399813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:13.399813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:13.399813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:13.399813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:13.399813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:13.399813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:13.399813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:13.399813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:13.399813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:13.399813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:22:13.399813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:13.399813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 407/815] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:13 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:13.400645 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbe3bf-e1c2-4e37-b50e-c03711808f1f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:13.488894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:13.506172 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbe3bf-e1c2-4e37-b50e-c03711808f1f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:13.508503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05b773-3bf0-486e-a25b-52fd2bfb6199 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:13.529185 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:13.543402 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c4ef3-3796-4da7-9215-69f72a2719c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:13.544171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c4ef3-3796-4da7-9215-69f72a2719c2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:13.544463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c4ef3-3796-4da7-9215-69f72a2719c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:13.544850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c4ef3-3796-4da7-9215-69f72a2719c2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:13.546436 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c69f0-9faf-4c72-b8fe-003ef8d7baa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:13.546584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c4ef3-3796-4da7-9215-69f72a2719c2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:13.546666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c69f0-9faf-4c72-b8fe-003ef8d7baa9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:13.547011 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c69f0-9faf-4c72-b8fe-003ef8d7baa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:13.547349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c69f0-9faf-4c72-b8fe-003ef8d7baa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:13.547977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c4ef3-3796-4da7-9215-69f72a2719c2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:13.559484 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c69f0-9faf-4c72-b8fe-003ef8d7baa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:13.560933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c69f0-9faf-4c72-b8fe-003ef8d7baa9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-8a4c4ef3-3796-4da7-9215-69f72a2719c2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:13.561280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:13.562360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:13.563231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 408/816] 10.222.0.64 () {56 vars in 1129 bytes} [Mon May 23 15:22:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:13.564093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c4ef3-3796-4da7-9215-69f72a2719c2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:13.582957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-ab3c69f0-9faf-4c72-b8fe-003ef8d7baa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:13.582957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:13.582957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:13.582957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:13.582957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:13.582957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:13.582957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:13.582957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:13.582957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:13.582957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:13.582957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:13.582957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:13.582957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:13.582957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:13.582957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:13.584104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:13.590048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:13.590048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:13.590048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:13.590048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:13.590048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 409/817] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:13.590048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c69f0-9faf-4c72-b8fe-003ef8d7baa9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:13.683178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c4ef3-3796-4da7-9215-69f72a2719c2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:13.693247 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c69f0-9faf-4c72-b8fe-003ef8d7baa9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:13.723244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56806339-e61d-4aa7-8a41-9e2f294e0bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:13.724069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56806339-e61d-4aa7-8a41-9e2f294e0bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:13.724353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56806339-e61d-4aa7-8a41-9e2f294e0bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:13.724632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56806339-e61d-4aa7-8a41-9e2f294e0bd1 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:13.725373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56806339-e61d-4aa7-8a41-9e2f294e0bd1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:13.725687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56806339-e61d-4aa7-8a41-9e2f294e0bd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:13.727061 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56806339-e61d-4aa7-8a41-9e2f294e0bd1 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:13.729933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:13.731998 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 409/818] 10.222.0.64 () {56 vars in 1112 bytes} [Mon May 23 15:22:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:13.732665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56806339-e61d-4aa7-8a41-9e2f294e0bd1 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:13.739797 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa72a7-1d3e-4607-adcc-64de601af81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:13.740674 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa72a7-1d3e-4607-adcc-64de601af81a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:13.741026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa72a7-1d3e-4607-adcc-64de601af81a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:13.742040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa72a7-1d3e-4607-adcc-64de601af81a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:13.752816 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa72a7-1d3e-4607-adcc-64de601af81a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:13.754532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa72a7-1d3e-4607-adcc-64de601af81a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:13.773733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-05fa72a7-1d3e-4607-adcc-64de601af81a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:13.773733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:13.773733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:13.773733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:13.773733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:13.773733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:13.773733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:13.773733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:13.773733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:13.773733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:13.773733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:13.773733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:13.773733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:13.773733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:13.773733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:13.774941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:13.776045 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:13.776045 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:13.776045 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:13.776045 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:13.776630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 410/819] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:13.777492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa72a7-1d3e-4607-adcc-64de601af81a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:13.790226 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:13.843978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56806339-e61d-4aa7-8a41-9e2f294e0bd1 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:13.879649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d839b-4446-4ac5-89c6-5dc7ba43fca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:13.880479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d839b-4446-4ac5-89c6-5dc7ba43fca9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:13.880845 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d839b-4446-4ac5-89c6-5dc7ba43fca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:13.881243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d839b-4446-4ac5-89c6-5dc7ba43fca9 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:13.882044 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d839b-4446-4ac5-89c6-5dc7ba43fca9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:13.882429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d839b-4446-4ac5-89c6-5dc7ba43fca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:13.883817 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d839b-4446-4ac5-89c6-5dc7ba43fca9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:13.884647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa72a7-1d3e-4607-adcc-64de601af81a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:13.889305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 410/820] 10.222.0.64 () {56 vars in 1124 bytes} [Mon May 23 15:22:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:13.890278 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d839b-4446-4ac5-89c6-5dc7ba43fca9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:13.921491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed5f84-b841-412b-9928-12b5eb3fc95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:13.922188 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed5f84-b841-412b-9928-12b5eb3fc95e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:13.922478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed5f84-b841-412b-9928-12b5eb3fc95e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:13.922939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed5f84-b841-412b-9928-12b5eb3fc95e None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:13.923547 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed5f84-b841-412b-9928-12b5eb3fc95e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:13.923837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed5f84-b841-412b-9928-12b5eb3fc95e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:13.925263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed5f84-b841-412b-9928-12b5eb3fc95e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:13.942728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 411/821] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:13.942728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed5f84-b841-412b-9928-12b5eb3fc95e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:13.988863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:13.999940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d839b-4446-4ac5-89c6-5dc7ba43fca9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:14.013285 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:14.035420 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad3ff7-d6eb-48d1-be8c-fabe09408c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:14.036092 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad3ff7-d6eb-48d1-be8c-fabe09408c33 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:14.036383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad3ff7-d6eb-48d1-be8c-fabe09408c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:14.036668 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad3ff7-d6eb-48d1-be8c-fabe09408c33 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:14.037421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad3ff7-d6eb-48d1-be8c-fabe09408c33 None admin] RBAC: Adding query filter params (type=network) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:14.037736 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad3ff7-d6eb-48d1-be8c-fabe09408c33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:14.039118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad3ff7-d6eb-48d1-be8c-fabe09408c33 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:14.048594 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 411/822] 10.222.0.64 () {56 vars in 1112 bytes} [Mon May 23 15:22:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:14.048594 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad3ff7-d6eb-48d1-be8c-fabe09408c33 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:14.051069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed5f84-b841-412b-9928-12b5eb3fc95e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:14.084299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241d1a9-9707-470b-b1d0-45abc016c897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:14.085051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241d1a9-9707-470b-b1d0-45abc016c897 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:14.085291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241d1a9-9707-470b-b1d0-45abc016c897 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:14.085574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241d1a9-9707-470b-b1d0-45abc016c897 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:14.086295 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241d1a9-9707-470b-b1d0-45abc016c897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:14.086609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241d1a9-9707-470b-b1d0-45abc016c897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:14.087973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241d1a9-9707-470b-b1d0-45abc016c897 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:14.100696 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 412/823] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:14.101398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241d1a9-9707-470b-b1d0-45abc016c897 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:14.156898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad3ff7-d6eb-48d1-be8c-fabe09408c33 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:14.196497 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29357c9e-3971-4553-a6f3-d9cefde6afa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:14.197340 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29357c9e-3971-4553-a6f3-d9cefde6afa9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:14.197759 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29357c9e-3971-4553-a6f3-d9cefde6afa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:14.198135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29357c9e-3971-4553-a6f3-d9cefde6afa9 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:14.198996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29357c9e-3971-4553-a6f3-d9cefde6afa9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:14.199420 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29357c9e-3971-4553-a6f3-d9cefde6afa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:14.201013 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29357c9e-3971-4553-a6f3-d9cefde6afa9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:14.211271 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 412/824] 10.222.0.64 () {56 vars in 1124 bytes} [Mon May 23 15:22:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:14.212275 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29357c9e-3971-4553-a6f3-d9cefde6afa9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:14.213468 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241d1a9-9707-470b-b1d0-45abc016c897 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:14.217178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:14.253093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e17cf-6483-4d79-b35e-9108a1739bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:14.253947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e17cf-6483-4d79-b35e-9108a1739bbd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:14.254238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e17cf-6483-4d79-b35e-9108a1739bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:14.254518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e17cf-6483-4d79-b35e-9108a1739bbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:14.261511 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e17cf-6483-4d79-b35e-9108a1739bbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:14.261896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:14.262955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e17cf-6483-4d79-b35e-9108a1739bbd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:14.277419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-667e17cf-6483-4d79-b35e-9108a1739bbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:14.277419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:14.277419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:14.277419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:14.277419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:14.277419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:14.277419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:14.277419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:14.277419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:14.277419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:14.277419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:14.277419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:14.277419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:14.277419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:14.277419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:14.278356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:14.279260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:14.279260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:14.279260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:14.279260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:14.279260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:14.279260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:14.279260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:14.279260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:14.279260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 413/825] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:14.279780 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e17cf-6483-4d79-b35e-9108a1739bbd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:14.327612 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29357c9e-3971-4553-a6f3-d9cefde6afa9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:14.363700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f1cf4-cf91-4f16-ae48-4a303f2208d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:14.364378 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f1cf4-cf91-4f16-ae48-4a303f2208d3 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:14.364734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f1cf4-cf91-4f16-ae48-4a303f2208d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:14.365108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f1cf4-cf91-4f16-ae48-4a303f2208d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:14.365939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f1cf4-cf91-4f16-ae48-4a303f2208d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:14.367407 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f1cf4-cf91-4f16-ae48-4a303f2208d3 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:14.389260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e17cf-6483-4d79-b35e-9108a1739bbd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:14.422031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83a987-d7f4-46e1-a726-367637b5eafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:14.422803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83a987-d7f4-46e1-a726-367637b5eafd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:14.423086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83a987-d7f4-46e1-a726-367637b5eafd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:14.423362 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83a987-d7f4-46e1-a726-367637b5eafd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:14.430724 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83a987-d7f4-46e1-a726-367637b5eafd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:14.432142 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83a987-d7f4-46e1-a726-367637b5eafd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:14.447346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-9f83a987-d7f4-46e1-a726-367637b5eafd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:14.447346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:14.447346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:14.447346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:14.447346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:14.447346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:14.447346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:14.447346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:14.447346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:14.447346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:14.447346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:14.447346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:14.447346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:14.447346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:14.447346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:14.452228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:14.453524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:14.453524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:14.453524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:14.453524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:14.453524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:14.453524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:14.453524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:14.453524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:14.453524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 414/826] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:14.453524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83a987-d7f4-46e1-a726-367637b5eafd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:14.478799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:14.482303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 413/827] 10.222.0.64 () {60 vars in 1144 bytes} [Mon May 23 15:22:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:22:14.483418 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f1cf4-cf91-4f16-ae48-4a303f2208d3 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:14.569523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83a987-d7f4-46e1-a726-367637b5eafd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:14.618063 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddb53f-ab9a-4fc5-9769-27d3345c286e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:14.618729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddb53f-ab9a-4fc5-9769-27d3345c286e None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:14.619121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddb53f-ab9a-4fc5-9769-27d3345c286e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:14.619499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddb53f-ab9a-4fc5-9769-27d3345c286e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:14.620360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddb53f-ab9a-4fc5-9769-27d3345c286e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:14.621960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddb53f-ab9a-4fc5-9769-27d3345c286e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:14.631670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:14.640325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f1cf4-cf91-4f16-ae48-4a303f2208d3 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:14.682625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19723311-c534-4fc0-b1ac-44d603c05920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:14.683310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19723311-c534-4fc0-b1ac-44d603c05920 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:14.683602 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19723311-c534-4fc0-b1ac-44d603c05920 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:14.683887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19723311-c534-4fc0-b1ac-44d603c05920 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:14.684690 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19723311-c534-4fc0-b1ac-44d603c05920 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:14.685005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19723311-c534-4fc0-b1ac-44d603c05920 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:14.686594 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19723311-c534-4fc0-b1ac-44d603c05920 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:14.702068 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 414/828] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:14.703030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19723311-c534-4fc0-b1ac-44d603c05920 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:14.730035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 415/829] 10.222.0.64 () {60 vars in 1144 bytes} [Mon May 23 15:22:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:22:14.731339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddb53f-ab9a-4fc5-9769-27d3345c286e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:14.841409 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19723311-c534-4fc0-b1ac-44d603c05920 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:14.861943 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:14.898722 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89646a5-35bd-4510-88e7-48274cb2cc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:14.899332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89646a5-35bd-4510-88e7-48274cb2cc1a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:14.899624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89646a5-35bd-4510-88e7-48274cb2cc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:14.899910 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89646a5-35bd-4510-88e7-48274cb2cc1a None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:14.900725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89646a5-35bd-4510-88e7-48274cb2cc1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:14.901045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89646a5-35bd-4510-88e7-48274cb2cc1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:14.902966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89646a5-35bd-4510-88e7-48274cb2cc1a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:14.903561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddb53f-ab9a-4fc5-9769-27d3345c286e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:14.925161 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 415/830] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:14.930361 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89646a5-35bd-4510-88e7-48274cb2cc1a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:14.946636 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:14.960344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829735c1-ddd5-4418-b9ef-84934ca1c609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:14.961512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829735c1-ddd5-4418-b9ef-84934ca1c609 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:14.964189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829735c1-ddd5-4418-b9ef-84934ca1c609 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:14.964189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829735c1-ddd5-4418-b9ef-84934ca1c609 None admin] PATH_INFO: `/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/08dad0888c8542f4aed748a1a35db2c4/roles/3d6e986dd2264e3cb7e01e25dbac6f97` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:14.982041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829735c1-ddd5-4418-b9ef-84934ca1c609 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d81eafac474f37a71ee5cc292d172f, user_id=08dad0888c8542f4aed748a1a35db2c4, role_id=3d6e986dd2264e3cb7e01e25dbac6f97)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:14.983503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829735c1-ddd5-4418-b9ef-84934ca1c609 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:15.049849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89646a5-35bd-4510-88e7-48274cb2cc1a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:15.061095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 416/831] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:22:14 2022] PUT /identity/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/08dad0888c8542f4aed748a1a35db2c4/roles/3d6e986dd2264e3cb7e01e25dbac6f97 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:22:15.096494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5d778-2d44-4f19-85de-04794bb9efa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:15.097307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5d778-2d44-4f19-85de-04794bb9efa9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:15.097595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5d778-2d44-4f19-85de-04794bb9efa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:15.097871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5d778-2d44-4f19-85de-04794bb9efa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:15.098839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5d778-2d44-4f19-85de-04794bb9efa9 None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=8da96bed73794fee8f719ec904936164) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:15.099155 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5d778-2d44-4f19-85de-04794bb9efa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:15.100858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5d778-2d44-4f19-85de-04794bb9efa9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:15.108248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 416/832] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:22:14 2022] GET /identity/v3/role_assignments?user.id=8da96bed73794fee8f719ec904936164&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:15.384062 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95cb4c-4e83-4a2f-bdb7-f00d3e72396c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:15.384597 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95cb4c-4e83-4a2f-bdb7-f00d3e72396c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:15.384597 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95cb4c-4e83-4a2f-bdb7-f00d3e72396c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:15.386177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 417/833] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:15.391999 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81230e5d-4230-4674-aaf4-519ca909c548 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:15.392305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81230e5d-4230-4674-aaf4-519ca909c548 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:15.392660 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81230e5d-4230-4674-aaf4-519ca909c548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:15.410402 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-81230e5d-4230-4674-aaf4-519ca909c548 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:15.411610 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-81230e5d-4230-4674-aaf4-519ca909c548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:15.482334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 417/834] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:15.506735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95cb4c-4e83-4a2f-bdb7-f00d3e72396c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:15.618941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95cb4c-4e83-4a2f-bdb7-f00d3e72396c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:15.644100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:15.659074 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dda42d-032c-4e7b-88b0-d539d6c9e503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:15.660245 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dda42d-032c-4e7b-88b0-d539d6c9e503 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:15.660739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dda42d-032c-4e7b-88b0-d539d6c9e503 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:15.661262 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dda42d-032c-4e7b-88b0-d539d6c9e503 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:15.672771 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dda42d-032c-4e7b-88b0-d539d6c9e503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-83dda42d-032c-4e7b-88b0-d539d6c9e503 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:15.682433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:15.683401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:15.684282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:15.684282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:15.684282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:15.684282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:15.684282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:15.684282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:15.684282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:15.684282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:15.684282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:15.684282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:15.684282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:15.684282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:15.691735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 418/835] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:22:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:15.693553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81230e5d-4230-4674-aaf4-519ca909c548 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:15.721440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:15.805719 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81230e5d-4230-4674-aaf4-519ca909c548 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:15.841310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e37a8-d9cb-446e-aa8f-33dbfdb48f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:15.841982 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e37a8-d9cb-446e-aa8f-33dbfdb48f9f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:15.842266 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e37a8-d9cb-446e-aa8f-33dbfdb48f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:15.842543 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e37a8-d9cb-446e-aa8f-33dbfdb48f9f None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:15.843284 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e37a8-d9cb-446e-aa8f-33dbfdb48f9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:15.843596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e37a8-d9cb-446e-aa8f-33dbfdb48f9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:15.844963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e37a8-d9cb-446e-aa8f-33dbfdb48f9f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:15.854396 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 418/836] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:22:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:15.862162 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dda42d-032c-4e7b-88b0-d539d6c9e503 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:15.969884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dda42d-032c-4e7b-88b0-d539d6c9e503 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:16.006008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77834a54-3b63-4b72-81ba-8905c3336928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:16.007163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77834a54-3b63-4b72-81ba-8905c3336928 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:16.007681 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77834a54-3b63-4b72-81ba-8905c3336928 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:16.008174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77834a54-3b63-4b72-81ba-8905c3336928 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:16.020418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77834a54-3b63-4b72-81ba-8905c3336928 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:16.022165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77834a54-3b63-4b72-81ba-8905c3336928 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:16.039984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-77834a54-3b63-4b72-81ba-8905c3336928 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:16.039984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:16.039984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:16.039984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:16.039984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:16.039984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:16.039984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:16.039984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:16.039984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:16.039984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:16.039984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:16.039984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:16.039984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:16.039984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:16.039984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:16.040942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:16.041883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:16.041883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:16.041883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:16.041883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:16.048412 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 419/837] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:16.050291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e37a8-d9cb-446e-aa8f-33dbfdb48f9f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:16.124751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:16.160795 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e37a8-d9cb-446e-aa8f-33dbfdb48f9f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:16.204538 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e221195-842f-410f-be41-83259616079e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:16.205850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e221195-842f-410f-be41-83259616079e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:16.206137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e221195-842f-410f-be41-83259616079e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:16.206416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e221195-842f-410f-be41-83259616079e None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:16.207135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e221195-842f-410f-be41-83259616079e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:16.207444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e221195-842f-410f-be41-83259616079e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:16.208968 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e221195-842f-410f-be41-83259616079e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:16.221535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 419/838] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:16.227013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77834a54-3b63-4b72-81ba-8905c3336928 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:16.265086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:16.336211 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77834a54-3b63-4b72-81ba-8905c3336928 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:16.372286 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acac62-a801-44ef-b11e-28c0cfa0b6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:16.373579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acac62-a801-44ef-b11e-28c0cfa0b6cc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:16.374084 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acac62-a801-44ef-b11e-28c0cfa0b6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:16.374572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acac62-a801-44ef-b11e-28c0cfa0b6cc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:16.383996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acac62-a801-44ef-b11e-28c0cfa0b6cc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:16.385839 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acac62-a801-44ef-b11e-28c0cfa0b6cc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-83acac62-a801-44ef-b11e-28c0cfa0b6cc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:22:16.402066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:16.403050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:16.404018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:16.404018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:16.404018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:22:16.404018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:22:16.404018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:22:16.404018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:22:16.404018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:22:16.404018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:22:16.404018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:22:16.404018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:16.410680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 420/839] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:22:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:16.413429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e221195-842f-410f-be41-83259616079e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:16.521123 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e221195-842f-410f-be41-83259616079e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:16.548303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:16.557539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf5a9d-0ed8-45f9-8d71-9bf7f9ae0f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:16.558215 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf5a9d-0ed8-45f9-8d71-9bf7f9ae0f37 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:16.558499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf5a9d-0ed8-45f9-8d71-9bf7f9ae0f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:16.558776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf5a9d-0ed8-45f9-8d71-9bf7f9ae0f37 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:16.559613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf5a9d-0ed8-45f9-8d71-9bf7f9ae0f37 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:16.559922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf5a9d-0ed8-45f9-8d71-9bf7f9ae0f37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:16.561375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf5a9d-0ed8-45f9-8d71-9bf7f9ae0f37 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:16.579106 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 420/840] 10.222.0.64 () {56 vars in 1135 bytes} [Mon May 23 15:22:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:16.585572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acac62-a801-44ef-b11e-28c0cfa0b6cc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:16.686238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:16.696364 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acac62-a801-44ef-b11e-28c0cfa0b6cc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:16.730375 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e868a0-93bf-43a9-afe7-6a9183855bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:16.731037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e868a0-93bf-43a9-afe7-6a9183855bae None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:16.731326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e868a0-93bf-43a9-afe7-6a9183855bae None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:16.731607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e868a0-93bf-43a9-afe7-6a9183855bae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:16.801611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e868a0-93bf-43a9-afe7-6a9183855bae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:16.803329 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e868a0-93bf-43a9-afe7-6a9183855bae None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:16.902314 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:16.915306 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 421/841] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:22:16 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:22:16.916684 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf5a9d-0ed8-45f9-8d71-9bf7f9ae0f37 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:17.023926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf5a9d-0ed8-45f9-8d71-9bf7f9ae0f37 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:17.062203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dfaa3-a907-402b-a2d8-6c5e87ec9f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:17.062957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dfaa3-a907-402b-a2d8-6c5e87ec9f0c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:17.063220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dfaa3-a907-402b-a2d8-6c5e87ec9f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:17.063504 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dfaa3-a907-402b-a2d8-6c5e87ec9f0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:17.075948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bc978-5ab2-4f5b-9467-70355839651b None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:17.076245 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bc978-5ab2-4f5b-9467-70355839651b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:17.076519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bc978-5ab2-4f5b-9467-70355839651b None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:17.078179 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4dfaa3-a907-402b-a2d8-6c5e87ec9f0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:17.078372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 422/842] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:17.080094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4dfaa3-a907-402b-a2d8-6c5e87ec9f0c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:17.085897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136b887-9bee-462b-aa06-b4796e0f2ab7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:17.086193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136b887-9bee-462b-aa06-b4796e0f2ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:17.086483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136b887-9bee-462b-aa06-b4796e0f2ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:17.102603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-bd4dfaa3-a907-402b-a2d8-6c5e87ec9f0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:17.102603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:17.102603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:17.102603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:17.102603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:17.102603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:17.102603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:17.102603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:17.102603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:17.102603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:17.102603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:17.102603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:17.102603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:17.102603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:17.102603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:17.103576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:17.104509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:17.104509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:17.104509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:17.104509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:17.104737 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6136b887-9bee-462b-aa06-b4796e0f2ab7 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:17.104737 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6136b887-9bee-462b-aa06-b4796e0f2ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:17.112839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 421/843] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:17.116393 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dfaa3-a907-402b-a2d8-6c5e87ec9f0c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:17.162071 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:17.197220 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 423/844] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:17.218823 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136b887-9bee-462b-aa06-b4796e0f2ab7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:17.246117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dfaa3-a907-402b-a2d8-6c5e87ec9f0c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:17.292497 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eccdfa-32fe-4660-83f0-da8c2b64d42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:17.293214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eccdfa-32fe-4660-83f0-da8c2b64d42e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:17.293647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eccdfa-32fe-4660-83f0-da8c2b64d42e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:17.294067 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eccdfa-32fe-4660-83f0-da8c2b64d42e None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:17.294812 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eccdfa-32fe-4660-83f0-da8c2b64d42e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:17.295127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eccdfa-32fe-4660-83f0-da8c2b64d42e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:17.297536 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eccdfa-32fe-4660-83f0-da8c2b64d42e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:17.316718 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 422/845] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:17.318814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eccdfa-32fe-4660-83f0-da8c2b64d42e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:17.338309 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136b887-9bee-462b-aa06-b4796e0f2ab7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:17.370345 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:17.383360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee0605-dfa7-47ba-b56d-a9cc12ab0ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:17.384191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee0605-dfa7-47ba-b56d-a9cc12ab0ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:17.384482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee0605-dfa7-47ba-b56d-a9cc12ab0ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:17.384761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee0605-dfa7-47ba-b56d-a9cc12ab0ab7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:17.385466 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee0605-dfa7-47ba-b56d-a9cc12ab0ab7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:17.390092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee0605-dfa7-47ba-b56d-a9cc12ab0ab7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-44ee0605-dfa7-47ba-b56d-a9cc12ab0ab7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:17.399894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:17.401006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:17.402327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:17.402327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:17.402327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:17.402327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:17.402327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:17.402327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:17.402327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:17.402327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:17.402327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:17.402327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:17.402327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:17.402327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:17.402327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:22:17.402327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:17.402327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 424/846] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:22:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:17.403168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c382f7-11d1-44b6-9f8f-be190351333c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:17.403168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c382f7-11d1-44b6-9f8f-be190351333c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:17.403168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c382f7-11d1-44b6-9f8f-be190351333c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:17.404353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 425/847] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:17.410392 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c382f7-11d1-44b6-9f8f-be190351333c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:17.438857 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:17.442554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eccdfa-32fe-4660-83f0-da8c2b64d42e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:17.477828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7dc30-248a-4057-85a3-a45d38555a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:17.478958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7dc30-248a-4057-85a3-a45d38555a42 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:17.479262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7dc30-248a-4057-85a3-a45d38555a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:17.479544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7dc30-248a-4057-85a3-a45d38555a42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:17.488078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7dc30-248a-4057-85a3-a45d38555a42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:17.489584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7dc30-248a-4057-85a3-a45d38555a42 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:17.503376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-7ff7dc30-248a-4057-85a3-a45d38555a42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:17.503376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:17.503376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:17.503376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:17.503376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:17.503376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:17.503376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:17.503376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:17.503376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:17.503376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:17.503376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:17.503376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:17.503376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:17.503376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:17.503376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:17.504310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:17.505250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:17.505250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:17.505250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:17.505250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:17.505250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:17.505250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:17.505250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:17.505250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:17.505250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 423/848] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:17.506496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144bcdc-5e1b-42dd-b188-4a3aa5b17623 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:17.506496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144bcdc-5e1b-42dd-b188-4a3aa5b17623 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:17.506496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144bcdc-5e1b-42dd-b188-4a3aa5b17623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:17.526241 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c382f7-11d1-44b6-9f8f-be190351333c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:17.526739 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c144bcdc-5e1b-42dd-b188-4a3aa5b17623 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:17.528369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c144bcdc-5e1b-42dd-b188-4a3aa5b17623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:17.561331 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267e8d3-3ac5-4fb4-93d2-c9bd1689582a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:17.562004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267e8d3-3ac5-4fb4-93d2-c9bd1689582a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:17.562295 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267e8d3-3ac5-4fb4-93d2-c9bd1689582a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:17.562575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267e8d3-3ac5-4fb4-93d2-c9bd1689582a None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:17.563279 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267e8d3-3ac5-4fb4-93d2-c9bd1689582a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:17.563593 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267e8d3-3ac5-4fb4-93d2-c9bd1689582a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:17.565338 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267e8d3-3ac5-4fb4-93d2-c9bd1689582a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:17.569940 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 426/849] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:22:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:17.570664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267e8d3-3ac5-4fb4-93d2-c9bd1689582a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:17.600601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 424/850] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:17.602034 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedbd8b-a78c-46e4-8666-17ff3005ae60 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:17.602252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedbd8b-a78c-46e4-8666-17ff3005ae60 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:17.602540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedbd8b-a78c-46e4-8666-17ff3005ae60 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:17.605380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 425/851] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:17.605380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedbd8b-a78c-46e4-8666-17ff3005ae60 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:17.630359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:17.676984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267e8d3-3ac5-4fb4-93d2-c9bd1689582a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:17.693774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:17.707325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedbd8b-a78c-46e4-8666-17ff3005ae60 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:17.711436 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1ee65-31c3-4ac1-9168-855f4f209870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:17.712101 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1ee65-31c3-4ac1-9168-855f4f209870 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:17.712386 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1ee65-31c3-4ac1-9168-855f4f209870 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:17.712659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1ee65-31c3-4ac1-9168-855f4f209870 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:17.713479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1ee65-31c3-4ac1-9168-855f4f209870 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:17.713789 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1ee65-31c3-4ac1-9168-855f4f209870 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:17.715200 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1ee65-31c3-4ac1-9168-855f4f209870 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:17.728064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 427/852] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:17.729392 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26b6ad-5226-4326-933e-0a3403c643f5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:17.729671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26b6ad-5226-4326-933e-0a3403c643f5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:17.729948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26b6ad-5226-4326-933e-0a3403c643f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:17.743019 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdab29-9524-4139-ba48-3304a7b22a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:17.743696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdab29-9524-4139-ba48-3304a7b22a0e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:17.743987 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdab29-9524-4139-ba48-3304a7b22a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:17.744270 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdab29-9524-4139-ba48-3304a7b22a0e None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:17.744979 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdab29-9524-4139-ba48-3304a7b22a0e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:17.745327 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdab29-9524-4139-ba48-3304a7b22a0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:17.745745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5c26b6ad-5226-4326-933e-0a3403c643f5 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:17.746716 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdab29-9524-4139-ba48-3304a7b22a0e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:17.746913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5c26b6ad-5226-4326-933e-0a3403c643f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:17.751737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 426/853] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:22:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:17.752517 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdab29-9524-4139-ba48-3304a7b22a0e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:17.823377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 428/854] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:17.824208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26b6ad-5226-4326-933e-0a3403c643f5 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:17.876832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdab29-9524-4139-ba48-3304a7b22a0e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:17.879359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:17.918332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d167e-4ca6-4d52-b711-b2e74d497c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:17.919331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d167e-4ca6-4d52-b711-b2e74d497c15 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:17.919624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d167e-4ca6-4d52-b711-b2e74d497c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:17.919907 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d167e-4ca6-4d52-b711-b2e74d497c15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:17.931810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d167e-4ca6-4d52-b711-b2e74d497c15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-861d167e-4ca6-4d52-b711-b2e74d497c15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:17.940324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:17.941312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:17.942224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:17.942224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:17.942224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:17.942224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:17.942224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:17.942224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:17.942224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:17.942224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:17.942224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:17.942224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:17.942224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:22:17.942224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:17.942224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 427/855] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:22:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:17.943864 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d167e-4ca6-4d52-b711-b2e74d497c15 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:17.948898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26b6ad-5226-4326-933e-0a3403c643f5 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:17.959987 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:17.985652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f1279-d5d4-47fd-9441-5d09e25e6f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:17.986620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f1279-d5d4-47fd-9441-5d09e25e6f4c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:17.986913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f1279-d5d4-47fd-9441-5d09e25e6f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:17.987198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f1279-d5d4-47fd-9441-5d09e25e6f4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:17.988035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f1279-d5d4-47fd-9441-5d09e25e6f4c None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=f8c79c36c7204368816593f0e473191b) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:17.988355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f1279-d5d4-47fd-9441-5d09e25e6f4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:17.989871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f1279-d5d4-47fd-9441-5d09e25e6f4c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:17.995902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 429/856] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:22:17 2022] GET /identity/v3/role_assignments?user.id=f8c79c36c7204368816593f0e473191b&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:17.996697 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f1279-d5d4-47fd-9441-5d09e25e6f4c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:18.069782 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d167e-4ca6-4d52-b711-b2e74d497c15 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:18.121048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f1279-d5d4-47fd-9441-5d09e25e6f4c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:18.124897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be689beb-408d-44f9-a57e-228c92558e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:18.125628 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be689beb-408d-44f9-a57e-228c92558e11 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:18.125916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be689beb-408d-44f9-a57e-228c92558e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:18.126197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be689beb-408d-44f9-a57e-228c92558e11 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:18.127415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be689beb-408d-44f9-a57e-228c92558e11 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:18.128828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be689beb-408d-44f9-a57e-228c92558e11 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:18.135411 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-be689beb-408d-44f9-a57e-228c92558e11 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:18.141967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:18.143839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:18.144727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 428/857] 10.222.0.64 () {56 vars in 1132 bytes} [Mon May 23 15:22:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:18.149691 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be689beb-408d-44f9-a57e-228c92558e11 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:18.174219 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6958e1-41e7-4503-b3a7-5767f6ef463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:18.175413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6958e1-41e7-4503-b3a7-5767f6ef463d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:18.175922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6958e1-41e7-4503-b3a7-5767f6ef463d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:18.176421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6958e1-41e7-4503-b3a7-5767f6ef463d None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:18.177690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6958e1-41e7-4503-b3a7-5767f6ef463d None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:18.178209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6958e1-41e7-4503-b3a7-5767f6ef463d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:18.179874 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6958e1-41e7-4503-b3a7-5767f6ef463d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:18.200216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 430/858] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:22:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:18.201654 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6958e1-41e7-4503-b3a7-5767f6ef463d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:18.256572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:18.286441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be689beb-408d-44f9-a57e-228c92558e11 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:18.325831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6958e1-41e7-4503-b3a7-5767f6ef463d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:18.326908 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c144c7-2304-48ed-9153-9f562ebfc0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:18.327695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c144c7-2304-48ed-9153-9f562ebfc0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:18.328055 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c144c7-2304-48ed-9153-9f562ebfc0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:18.328412 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c144c7-2304-48ed-9153-9f562ebfc0f2 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:18.329279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c144c7-2304-48ed-9153-9f562ebfc0f2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:18.329690 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c144c7-2304-48ed-9153-9f562ebfc0f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:18.331869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c144c7-2304-48ed-9153-9f562ebfc0f2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:18.337443 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 429/859] 10.222.0.64 () {56 vars in 1126 bytes} [Mon May 23 15:22:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:18.342316 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c144c7-2304-48ed-9153-9f562ebfc0f2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:18.370307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494c0b6-3f7d-4879-a2e2-dbabb31ea059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:18.371181 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494c0b6-3f7d-4879-a2e2-dbabb31ea059 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:18.371413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494c0b6-3f7d-4879-a2e2-dbabb31ea059 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:18.371695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494c0b6-3f7d-4879-a2e2-dbabb31ea059 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:18.388880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494c0b6-3f7d-4879-a2e2-dbabb31ea059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:18.390360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494c0b6-3f7d-4879-a2e2-dbabb31ea059 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:18.410259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6494c0b6-3f7d-4879-a2e2-dbabb31ea059 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:18.410259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:18.410259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:18.410259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:18.410259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:18.410259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:18.410259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:18.410259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:18.410259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:18.410259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:18.410259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:18.410259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:18.410259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:18.410259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:18.410259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:18.411233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:18.412190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:18.412190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:18.412190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:18.412190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:18.416073 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 431/860] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:18.417285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494c0b6-3f7d-4879-a2e2-dbabb31ea059 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:18.442739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:18.482541 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c144c7-2304-48ed-9153-9f562ebfc0f2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:18.528133 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7a5dc-4ac6-4265-9c8c-8cbc5f2064ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:18.529138 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7a5dc-4ac6-4265-9c8c-8cbc5f2064ab None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:18.529608 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494c0b6-3f7d-4879-a2e2-dbabb31ea059 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:18.529815 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7a5dc-4ac6-4265-9c8c-8cbc5f2064ab None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:18.530267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7a5dc-4ac6-4265-9c8c-8cbc5f2064ab None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:18.531186 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7a5dc-4ac6-4265-9c8c-8cbc5f2064ab None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:18.531942 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7a5dc-4ac6-4265-9c8c-8cbc5f2064ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:18.538772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7a5dc-4ac6-4265-9c8c-8cbc5f2064ab None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:18.546010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 430/861] 10.222.0.64 () {56 vars in 1126 bytes} [Mon May 23 15:22:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:18.572248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7340c-2568-4092-bc49-cefc4a8e9d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:18.573146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7340c-2568-4092-bc49-cefc4a8e9d78 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:18.573502 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7340c-2568-4092-bc49-cefc4a8e9d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:18.573851 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7340c-2568-4092-bc49-cefc4a8e9d78 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:18.574732 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7340c-2568-4092-bc49-cefc4a8e9d78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:18.575167 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7340c-2568-4092-bc49-cefc4a8e9d78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:18.576730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7340c-2568-4092-bc49-cefc4a8e9d78 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:18.596655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 432/862] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:18.598138 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7a5dc-4ac6-4265-9c8c-8cbc5f2064ab None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:18.605503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:18.724753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7a5dc-4ac6-4265-9c8c-8cbc5f2064ab None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:18.762227 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bc78e-c295-4af0-9c90-53e833778283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:18.763050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bc78e-c295-4af0-9c90-53e833778283 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:18.763339 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bc78e-c295-4af0-9c90-53e833778283 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:18.763615 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bc78e-c295-4af0-9c90-53e833778283 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:18.772139 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bc78e-c295-4af0-9c90-53e833778283 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:18.773629 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bc78e-c295-4af0-9c90-53e833778283 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-613bc78e-c295-4af0-9c90-53e833778283 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:22:18.788963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:18.789988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:18.790946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:18.790946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:18.790946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:22:18.790946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:22:18.790946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:22:18.790946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:22:18.790946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:22:18.790946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:22:18.790946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:22:18.790946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:18.790946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 431/863] 10.222.0.64 () {56 vars in 1117 bytes} [Mon May 23 15:22:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:18.795946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7340c-2568-4092-bc49-cefc4a8e9d78 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:18.878803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:18.898084 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7340c-2568-4092-bc49-cefc4a8e9d78 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:18.935774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780563f9-59b2-4fe9-938d-b6856fa5ce64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:18.936446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780563f9-59b2-4fe9-938d-b6856fa5ce64 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:18.936734 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780563f9-59b2-4fe9-938d-b6856fa5ce64 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:18.937008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780563f9-59b2-4fe9-938d-b6856fa5ce64 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:18.937884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780563f9-59b2-4fe9-938d-b6856fa5ce64 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:18.938196 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780563f9-59b2-4fe9-938d-b6856fa5ce64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:18.940161 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780563f9-59b2-4fe9-938d-b6856fa5ce64 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:18.962738 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 433/864] 10.222.0.64 () {56 vars in 1149 bytes} [Mon May 23 15:22:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:18.963994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bc78e-c295-4af0-9c90-53e833778283 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:19.029253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:19.069262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bc78e-c295-4af0-9c90-53e833778283 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:19.104037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8beaaa-a22f-4077-98de-cb3a458ed234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:19.105529 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8beaaa-a22f-4077-98de-cb3a458ed234 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:19.106018 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8beaaa-a22f-4077-98de-cb3a458ed234 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:19.106494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8beaaa-a22f-4077-98de-cb3a458ed234 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:19.176490 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8beaaa-a22f-4077-98de-cb3a458ed234 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:19.178574 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8beaaa-a22f-4077-98de-cb3a458ed234 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:19.241044 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:19.285740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 432/865] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:22:18 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:22:19.293994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780563f9-59b2-4fe9-938d-b6856fa5ce64 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:19.396573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780563f9-59b2-4fe9-938d-b6856fa5ce64 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:19.432281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3ccc1-f2cb-46bc-8d18-8ac658b83385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:19.433099 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3ccc1-f2cb-46bc-8d18-8ac658b83385 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:19.433388 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3ccc1-f2cb-46bc-8d18-8ac658b83385 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:19.433662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3ccc1-f2cb-46bc-8d18-8ac658b83385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:19.440665 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:19.444402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3ccc1-f2cb-46bc-8d18-8ac658b83385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:19.446030 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3ccc1-f2cb-46bc-8d18-8ac658b83385 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:19.464632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-39b3ccc1-f2cb-46bc-8d18-8ac658b83385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:19.464632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:19.464632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:19.464632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:19.464632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:19.464632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:19.464632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:19.464632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:19.464632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:19.464632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:19.464632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:19.464632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:19.464632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:19.464632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:19.464632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:19.465600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:19.466509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:19.466509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:19.466509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:19.466509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:19.471096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 434/866] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:19.475282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8beaaa-a22f-4077-98de-cb3a458ed234 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:19.580170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8beaaa-a22f-4077-98de-cb3a458ed234 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:19.615919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791a2c0-fa1a-4b23-9388-4f4409c3c340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:19.616590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791a2c0-fa1a-4b23-9388-4f4409c3c340 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:19.616873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791a2c0-fa1a-4b23-9388-4f4409c3c340 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:19.617178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791a2c0-fa1a-4b23-9388-4f4409c3c340 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:19.617906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791a2c0-fa1a-4b23-9388-4f4409c3c340 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:19.618215 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791a2c0-fa1a-4b23-9388-4f4409c3c340 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:19.619703 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791a2c0-fa1a-4b23-9388-4f4409c3c340 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:19.636684 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 433/867] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:19.641930 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3ccc1-f2cb-46bc-8d18-8ac658b83385 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:19.757644 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3ccc1-f2cb-46bc-8d18-8ac658b83385 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:19.794645 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26bb09-0bc5-4a3c-b8de-5527873907e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:19.795502 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26bb09-0bc5-4a3c-b8de-5527873907e5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:19.795788 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26bb09-0bc5-4a3c-b8de-5527873907e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:19.796111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26bb09-0bc5-4a3c-b8de-5527873907e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:19.803442 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26bb09-0bc5-4a3c-b8de-5527873907e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:19.804875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26bb09-0bc5-4a3c-b8de-5527873907e5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:19.819537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-ee26bb09-0bc5-4a3c-b8de-5527873907e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:19.819537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:19.819537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:19.819537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:19.819537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:19.819537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:19.819537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:19.819537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:19.819537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:19.819537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:19.819537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:19.819537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:19.819537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:19.819537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:19.819537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:19.821613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:19.823702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:19.823702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:19.823702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:19.823702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:19.823702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:19.823702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:19.823702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:19.823702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:19.827426 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 435/868] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:19.827668 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791a2c0-fa1a-4b23-9388-4f4409c3c340 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:19.857921 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:19.935823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791a2c0-fa1a-4b23-9388-4f4409c3c340 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:19.971391 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe1072-ce4a-4e1e-82db-fc786cf58d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:19.972064 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe1072-ce4a-4e1e-82db-fc786cf58d52 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:19.972351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe1072-ce4a-4e1e-82db-fc786cf58d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:19.972631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe1072-ce4a-4e1e-82db-fc786cf58d52 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:19.973566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe1072-ce4a-4e1e-82db-fc786cf58d52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:19.973878 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe1072-ce4a-4e1e-82db-fc786cf58d52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:19.975351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe1072-ce4a-4e1e-82db-fc786cf58d52 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:19.992257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 434/869] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:19.993642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26bb09-0bc5-4a3c-b8de-5527873907e5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:20.033564 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec0eaa-3efe-464a-a1fd-6d231c880cb4 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:20.033933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec0eaa-3efe-464a-a1fd-6d231c880cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:20.034217 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec0eaa-3efe-464a-a1fd-6d231c880cb4 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:20.035857 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 435/870] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:20.041784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb5923-4733-48e6-a005-d97604ecb0e0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:20.042077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb5923-4733-48e6-a005-d97604ecb0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:20.042360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb5923-4733-48e6-a005-d97604ecb0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:20.057941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:20.061548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-eacb5923-4733-48e6-a005-d97604ecb0e0 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:20.062981 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-eacb5923-4733-48e6-a005-d97604ecb0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:20.111401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26bb09-0bc5-4a3c-b8de-5527873907e5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:20.141405 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 436/871] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:20.145574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f146dab-44b2-4012-a7a1-3549fdf5051d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:20.146229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f146dab-44b2-4012-a7a1-3549fdf5051d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:20.146513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f146dab-44b2-4012-a7a1-3549fdf5051d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:20.146801 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f146dab-44b2-4012-a7a1-3549fdf5051d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:20.147609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f146dab-44b2-4012-a7a1-3549fdf5051d None admin] RBAC: Adding query filter params (role.id=3d6e986dd2264e3cb7e01e25dbac6f97, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=08dad0888c8542f4aed748a1a35db2c4) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:20.147916 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f146dab-44b2-4012-a7a1-3549fdf5051d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:20.149352 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f146dab-44b2-4012-a7a1-3549fdf5051d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:20.159346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 436/872] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:22:19 2022] GET /identity/v3/role_assignments?user.id=08dad0888c8542f4aed748a1a35db2c4&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=3d6e986dd2264e3cb7e01e25dbac6f97 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:20.160621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb5923-4733-48e6-a005-d97604ecb0e0 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:20.203809 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:20.282472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb5923-4733-48e6-a005-d97604ecb0e0 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:20.322839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed1a82-4da0-4c24-b4f2-d1f952bc76aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:20.323935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed1a82-4da0-4c24-b4f2-d1f952bc76aa None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:20.324420 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed1a82-4da0-4c24-b4f2-d1f952bc76aa None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:20.324881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed1a82-4da0-4c24-b4f2-d1f952bc76aa None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:20.325844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed1a82-4da0-4c24-b4f2-d1f952bc76aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:20.327445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed1a82-4da0-4c24-b4f2-d1f952bc76aa None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:20.403473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 437/873] 10.222.0.64 () {60 vars in 1141 bytes} [Mon May 23 15:22:20 2022] POST /identity/v3/services => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:22:20.405934 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413d654-5985-4f4b-be19-ac813c0a768e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:20.406397 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413d654-5985-4f4b-be19-ac813c0a768e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:20.406577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413d654-5985-4f4b-be19-ac813c0a768e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:20.408922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 437/874] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:20.414770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b1cf3-fc75-4ed5-adb7-06e0f2357cdd None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:20.415049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b1cf3-fc75-4ed5-adb7-06e0f2357cdd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:20.415344 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b1cf3-fc75-4ed5-adb7-06e0f2357cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:20.435336 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-270b1cf3-fc75-4ed5-adb7-06e0f2357cdd None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:20.436734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-270b1cf3-fc75-4ed5-adb7-06e0f2357cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:20.518476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:20.524956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 438/875] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:20.547199 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413d654-5985-4f4b-be19-ac813c0a768e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:20.666041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413d654-5985-4f4b-be19-ac813c0a768e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:20.681414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:20.707828 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f9586-bb46-4e38-ac43-8681cec7e54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:20.708655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f9586-bb46-4e38-ac43-8681cec7e54a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:20.708940 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f9586-bb46-4e38-ac43-8681cec7e54a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:20.709249 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f9586-bb46-4e38-ac43-8681cec7e54a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:20.709907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f9586-bb46-4e38-ac43-8681cec7e54a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:20.711264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f9586-bb46-4e38-ac43-8681cec7e54a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-ca0f9586-bb46-4e38-ac43-8681cec7e54a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:20.723662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:20.724694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:20.725607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:20.725607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:20.725607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:20.725607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:20.725607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:20.725607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:20.725607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:20.725607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:20.725607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:20.725607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:20.725607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:20.725607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:20.725607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:22:20.725607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:20.725607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 438/876] 10.222.0.64 () {56 vars in 1126 bytes} [Mon May 23 15:22:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:20.732664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b1cf3-fc75-4ed5-adb7-06e0f2357cdd None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:20.842900 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b1cf3-fc75-4ed5-adb7-06e0f2357cdd None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:20.878967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b7c4b-5118-4e87-b08f-e917deff3b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:20.880069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b7c4b-5118-4e87-b08f-e917deff3b2a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:20.880565 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b7c4b-5118-4e87-b08f-e917deff3b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:20.881094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b7c4b-5118-4e87-b08f-e917deff3b2a None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:20.882013 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b7c4b-5118-4e87-b08f-e917deff3b2a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:20.882529 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b7c4b-5118-4e87-b08f-e917deff3b2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:20.884116 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b7c4b-5118-4e87-b08f-e917deff3b2a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:20.889474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 439/877] 10.222.0.64 () {56 vars in 1122 bytes} [Mon May 23 15:22:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:20.895962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f9586-bb46-4e38-ac43-8681cec7e54a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:20.942199 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fda1d-edf8-4fdf-8d2e-a3d2ff1e5fc3 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:20.942818 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fda1d-edf8-4fdf-8d2e-a3d2ff1e5fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:20.943329 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fda1d-edf8-4fdf-8d2e-a3d2ff1e5fc3 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:20.945233 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 440/878] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:20.950453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e2489-1e3a-4189-b65c-eba29b67ea58 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:20.951006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e2489-1e3a-4189-b65c-eba29b67ea58 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:20.951495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e2489-1e3a-4189-b65c-eba29b67ea58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:20.970009 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c87e2489-1e3a-4189-b65c-eba29b67ea58 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:20.972106 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c87e2489-1e3a-4189-b65c-eba29b67ea58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:21.005459 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f9586-bb46-4e38-ac43-8681cec7e54a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:21.038983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:21.043247 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c90bf-3351-446d-b093-05b432bdabb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:21.043921 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c90bf-3351-446d-b093-05b432bdabb7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:21.044207 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c90bf-3351-446d-b093-05b432bdabb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:21.044485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c90bf-3351-446d-b093-05b432bdabb7 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:21.045353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c90bf-3351-446d-b093-05b432bdabb7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:21.045654 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c90bf-3351-446d-b093-05b432bdabb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:21.047025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c90bf-3351-446d-b093-05b432bdabb7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:21.050262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 441/879] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:21.058108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 439/880] 10.222.0.64 () {56 vars in 1122 bytes} [Mon May 23 15:22:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:21.073447 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e2489-1e3a-4189-b65c-eba29b67ea58 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:21.187359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e2489-1e3a-4189-b65c-eba29b67ea58 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:21.227410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60d5ce-a03b-4eba-9193-d0f2f0cf3726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:21.228457 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60d5ce-a03b-4eba-9193-d0f2f0cf3726 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:21.228952 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60d5ce-a03b-4eba-9193-d0f2f0cf3726 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:21.229501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60d5ce-a03b-4eba-9193-d0f2f0cf3726 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:21.230462 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60d5ce-a03b-4eba-9193-d0f2f0cf3726 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:21.232080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60d5ce-a03b-4eba-9193-d0f2f0cf3726 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:21.340810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 442/881] 10.222.0.64 () {60 vars in 1141 bytes} [Mon May 23 15:22:21 2022] POST /identity/v3/services => generated 269 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:22:22.526889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc9e29-278b-4332-83c9-d30666df2616 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:22.527457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc9e29-278b-4332-83c9-d30666df2616 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:22.527457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc9e29-278b-4332-83c9-d30666df2616 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:22.529239 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 440/882] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:22.535963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5023d4-6b2e-45e9-ae6a-8c6aed063cc9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:22.536511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5023d4-6b2e-45e9-ae6a-8c6aed063cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:22.536511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5023d4-6b2e-45e9-ae6a-8c6aed063cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:22.554691 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-de5023d4-6b2e-45e9-ae6a-8c6aed063cc9 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:22.555907 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-de5023d4-6b2e-45e9-ae6a-8c6aed063cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:22.606561 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:22.637171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 443/883] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:22.653099 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc9e29-278b-4332-83c9-d30666df2616 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:22.706863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:22.721283 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6762f84-6dd9-45de-b300-0dd62ba6e5f3 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:22.721552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6762f84-6dd9-45de-b300-0dd62ba6e5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:22.721808 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6762f84-6dd9-45de-b300-0dd62ba6e5f3 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:22.723732 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 444/884] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:22.728896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49575a0-d3d8-4842-bdcc-c41be726cacd None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:22.729451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49575a0-d3d8-4842-bdcc-c41be726cacd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:22.729951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49575a0-d3d8-4842-bdcc-c41be726cacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:22.750936 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f49575a0-d3d8-4842-bdcc-c41be726cacd None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:22.752896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f49575a0-d3d8-4842-bdcc-c41be726cacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:22.768642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc9e29-278b-4332-83c9-d30666df2616 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:22.806503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d987b7-be3d-4071-8393-1e6ef1bb7c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:22.807332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d987b7-be3d-4071-8393-1e6ef1bb7c9e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:22.807621 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d987b7-be3d-4071-8393-1e6ef1bb7c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:22.807901 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d987b7-be3d-4071-8393-1e6ef1bb7c9e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:22.819615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d987b7-be3d-4071-8393-1e6ef1bb7c9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:22.823663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 445/885] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d0d987b7-be3d-4071-8393-1e6ef1bb7c9e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:22.828297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:22.829315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:22.830754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:22.830754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:22.830754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:22.830754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:22.830754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:22.830754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:22.830754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:22.830754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:22.830754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:22.830754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:22.830754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:22:22.830754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:22.833121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 441/886] 10.222.0.64 () {56 vars in 1120 bytes} [Mon May 23 15:22:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:22.838605 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49575a0-d3d8-4842-bdcc-c41be726cacd None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:22.845837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d987b7-be3d-4071-8393-1e6ef1bb7c9e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:22.948364 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49575a0-d3d8-4842-bdcc-c41be726cacd None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:22.960769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:22.974612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d987b7-be3d-4071-8393-1e6ef1bb7c9e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:22.976775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:22.986875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ba442-dcbe-4e73-83fb-d9501b3f0622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:22.987879 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ba442-dcbe-4e73-83fb-d9501b3f0622 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:22.988390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ba442-dcbe-4e73-83fb-d9501b3f0622 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:22.988892 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ba442-dcbe-4e73-83fb-d9501b3f0622 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:22.990721 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ba442-dcbe-4e73-83fb-d9501b3f0622 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:22.991249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ba442-dcbe-4e73-83fb-d9501b3f0622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:22.992834 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ba442-dcbe-4e73-83fb-d9501b3f0622 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:23.002566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 446/887] 10.222.0.64 () {56 vars in 1115 bytes} [Mon May 23 15:22:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:23.008750 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ba442-dcbe-4e73-83fb-d9501b3f0622 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:23.019702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a529b-e448-479e-b080-8192567e66b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:23.020886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a529b-e448-479e-b080-8192567e66b5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.021537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a529b-e448-479e-b080-8192567e66b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.022057 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a529b-e448-479e-b080-8192567e66b5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.023000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a529b-e448-479e-b080-8192567e66b5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:23.024727 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a529b-e448-479e-b080-8192567e66b5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-444a529b-e448-479e-b080-8192567e66b5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:23.039055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:23.040171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:23.041035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:23.041035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:23.041035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:23.041035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:23.041035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:23.041035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:23.041035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:23.041035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:23.041035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:23.041035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:23.041035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:23.041035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:23.041035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:22:23.041035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:23.048846 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 442/888] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:22:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:23.050171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a529b-e448-479e-b080-8192567e66b5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:23.132352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ba442-dcbe-4e73-83fb-d9501b3f0622 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:23.171649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a529b-e448-479e-b080-8192567e66b5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:23.177828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce5886-fd72-4478-a070-b2b4e8c2a467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:23.178663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce5886-fd72-4478-a070-b2b4e8c2a467 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.178946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce5886-fd72-4478-a070-b2b4e8c2a467 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.179219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce5886-fd72-4478-a070-b2b4e8c2a467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.195751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce5886-fd72-4478-a070-b2b4e8c2a467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:23.197788 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce5886-fd72-4478-a070-b2b4e8c2a467 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:23.217147 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef60726-e1f6-4c85-992d-7a5616b254d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:23.218317 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef60726-e1f6-4c85-992d-7a5616b254d5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.218828 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef60726-e1f6-4c85-992d-7a5616b254d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.219308 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef60726-e1f6-4c85-992d-7a5616b254d5 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.220413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef60726-e1f6-4c85-992d-7a5616b254d5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:23.221036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-dbce5886-fd72-4478-a070-b2b4e8c2a467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:23.221036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:23.221036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:23.221036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:23.221036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:23.221036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:23.221036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:23.221036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:23.221036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:23.221036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:23.221036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:23.221036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:23.221036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:23.221036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:23.221036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:23.221914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:23.222865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:23.222865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:23.222865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:23.222865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:23.223876 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef60726-e1f6-4c85-992d-7a5616b254d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:23.225551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef60726-e1f6-4c85-992d-7a5616b254d5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:23.226656 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 447/889] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:23.228716 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce5886-fd72-4478-a070-b2b4e8c2a467 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:23.237386 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 443/890] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:22:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:23.238927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef60726-e1f6-4c85-992d-7a5616b254d5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:23.259410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:23.275765 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:23.335121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce5886-fd72-4478-a070-b2b4e8c2a467 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:23.363753 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef60726-e1f6-4c85-992d-7a5616b254d5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:23.370503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6621e-baaa-413b-a8ca-a46479a64fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:23.371172 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6621e-baaa-413b-a8ca-a46479a64fab None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.371457 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6621e-baaa-413b-a8ca-a46479a64fab None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.371775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6621e-baaa-413b-a8ca-a46479a64fab None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.372449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6621e-baaa-413b-a8ca-a46479a64fab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:23.372756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6621e-baaa-413b-a8ca-a46479a64fab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:23.374163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6621e-baaa-413b-a8ca-a46479a64fab None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:23.392064 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 448/891] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:23.392064 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc7f5-0430-4f7c-bf18-3e0ba254069d None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.392064 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc7f5-0430-4f7c-bf18-3e0ba254069d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.392064 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc7f5-0430-4f7c-bf18-3e0ba254069d None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.393383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 449/892] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:23.394194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cc7f5-0430-4f7c-bf18-3e0ba254069d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:23.410409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2570efc-a97e-4f3d-a538-3c028faef9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:23.411445 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2570efc-a97e-4f3d-a538-3c028faef9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.411735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2570efc-a97e-4f3d-a538-3c028faef9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.412271 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2570efc-a97e-4f3d-a538-3c028faef9e6 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.412982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2570efc-a97e-4f3d-a538-3c028faef9e6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:23.413482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2570efc-a97e-4f3d-a538-3c028faef9e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:23.415129 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2570efc-a97e-4f3d-a538-3c028faef9e6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:23.427263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 444/893] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:22:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:23.427263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfd198-308d-4847-a76e-c386eee22daa None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.427263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfd198-308d-4847-a76e-c386eee22daa None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.427263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfd198-308d-4847-a76e-c386eee22daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.448767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-29cfd198-308d-4847-a76e-c386eee22daa None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:23.450106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-29cfd198-308d-4847-a76e-c386eee22daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:23.489707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:23.507683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cc7f5-0430-4f7c-bf18-3e0ba254069d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:23.525521 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 445/894] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:23.526378 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cfd198-308d-4847-a76e-c386eee22daa None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:23.549375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc9d01-263c-499f-b2f2-61c74ec26b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:23.550576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc9d01-263c-499f-b2f2-61c74ec26b8d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.550919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc9d01-263c-499f-b2f2-61c74ec26b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.551199 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc9d01-263c-499f-b2f2-61c74ec26b8d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.558990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc9d01-263c-499f-b2f2-61c74ec26b8d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:23.560494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc9d01-263c-499f-b2f2-61c74ec26b8d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:23.564473 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-78dc9d01-263c-499f-b2f2-61c74ec26b8d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:22:23.576665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:23.578195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:23.579369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:23.579369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:23.579369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:22:23.579369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:22:23.579369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:22:23.579369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:22:23.579369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:22:23.579369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:22:23.579369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:22:23.579369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:23.582666 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 450/895] 10.222.0.64 () {56 vars in 1117 bytes} [Mon May 23 15:22:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:23.583508 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc9d01-263c-499f-b2f2-61c74ec26b8d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:23.644975 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cfd198-308d-4847-a76e-c386eee22daa None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:23.685071 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b15ed-0510-44be-abc6-f9a34c629cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:23.685753 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b15ed-0510-44be-abc6-f9a34c629cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.686042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b15ed-0510-44be-abc6-f9a34c629cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.686329 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b15ed-0510-44be-abc6-f9a34c629cf9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.687112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b15ed-0510-44be-abc6-f9a34c629cf9 None admin] RBAC: Adding query filter params (interface=public, service_id=5351a52cdc4d405c8801cd01d7dfa738, region_id=RegionOne) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:23.687427 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b15ed-0510-44be-abc6-f9a34c629cf9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:23.688787 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b15ed-0510-44be-abc6-f9a34c629cf9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:23.689053 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc9d01-263c-499f-b2f2-61c74ec26b8d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:23.706060 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 446/896] 10.222.0.64 () {56 vars in 1251 bytes} [Mon May 23 15:22:23 2022] GET /identity/v3/endpoints?service_id=5351a52cdc4d405c8801cd01d7dfa738&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:23.706060 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b15ed-0510-44be-abc6-f9a34c629cf9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:23.727555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a638af8-50c9-4a3c-bd1f-b2120f571122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:23.728234 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a638af8-50c9-4a3c-bd1f-b2120f571122 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.728518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a638af8-50c9-4a3c-bd1f-b2120f571122 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.728796 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a638af8-50c9-4a3c-bd1f-b2120f571122 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.729572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a638af8-50c9-4a3c-bd1f-b2120f571122 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:23.730913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a638af8-50c9-4a3c-bd1f-b2120f571122 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:23.791292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:23.801988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 451/897] 10.222.0.64 () {60 vars in 1141 bytes} [Mon May 23 15:22:23 2022] POST /identity/v3/services => generated 241 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:22:23.803184 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b15ed-0510-44be-abc6-f9a34c629cf9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:23.803291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef56b47-82b8-46ee-b452-29991e70e7ab None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.803572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef56b47-82b8-46ee-b452-29991e70e7ab None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.803855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef56b47-82b8-46ee-b452-29991e70e7ab None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.810067 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 452/898] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:23.810067 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef56b47-82b8-46ee-b452-29991e70e7ab None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:23.810865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:23.855421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527bba2-3e47-48fc-b8d2-77da3b117023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:23.856790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527bba2-3e47-48fc-b8d2-77da3b117023 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.857283 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527bba2-3e47-48fc-b8d2-77da3b117023 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.857394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527bba2-3e47-48fc-b8d2-77da3b117023 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.858234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527bba2-3e47-48fc-b8d2-77da3b117023 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:23.861303 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527bba2-3e47-48fc-b8d2-77da3b117023 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:23.861303 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527bba2-3e47-48fc-b8d2-77da3b117023 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:23.879322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 447/899] 10.222.0.64 () {56 vars in 1149 bytes} [Mon May 23 15:22:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:23.879322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb61bce-521d-49c3-8f51-85841f2d8195 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.879322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb61bce-521d-49c3-8f51-85841f2d8195 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.879322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb61bce-521d-49c3-8f51-85841f2d8195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.899878 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3cb61bce-521d-49c3-8f51-85841f2d8195 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:23.901015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3cb61bce-521d-49c3-8f51-85841f2d8195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:23.925237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef56b47-82b8-46ee-b452-29991e70e7ab None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:23.963148 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17670893-7cad-4804-9d9c-f964ba752e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:23.963837 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17670893-7cad-4804-9d9c-f964ba752e80 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:23.964129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17670893-7cad-4804-9d9c-f964ba752e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:23.964427 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17670893-7cad-4804-9d9c-f964ba752e80 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:23.965116 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17670893-7cad-4804-9d9c-f964ba752e80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:23.966514 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17670893-7cad-4804-9d9c-f964ba752e80 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:23.981625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 453/900] 10.222.0.64 () {56 vars in 1087 bytes} [Mon May 23 15:22:23 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:22:23.981625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17670893-7cad-4804-9d9c-f964ba752e80 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:23.983439 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 448/901] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:24.006897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb61bce-521d-49c3-8f51-85841f2d8195 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:24.072214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:24.093509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:24.119062 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17670893-7cad-4804-9d9c-f964ba752e80 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:24.130294 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb61bce-521d-49c3-8f51-85841f2d8195 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:24.174639 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48587b-9ad5-490a-a704-8f43f7ed4d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:24.175244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48587b-9ad5-490a-a704-8f43f7ed4d2a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:24.175535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48587b-9ad5-490a-a704-8f43f7ed4d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:24.175826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48587b-9ad5-490a-a704-8f43f7ed4d2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:24.177413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76f916-d0ef-4709-a4f2-bc021fb1ea65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:24.178476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76f916-d0ef-4709-a4f2-bc021fb1ea65 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:24.178983 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76f916-d0ef-4709-a4f2-bc021fb1ea65 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:24.179262 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76f916-d0ef-4709-a4f2-bc021fb1ea65 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:24.180447 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76f916-d0ef-4709-a4f2-bc021fb1ea65 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:24.181831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76f916-d0ef-4709-a4f2-bc021fb1ea65 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-0a76f916-d0ef-4709-a4f2-bc021fb1ea65 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:24.198104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:24.199131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:24.200024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 449/902] 10.222.0.64 () {56 vars in 1132 bytes} [Mon May 23 15:22:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:24.207060 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76f916-d0ef-4709-a4f2-bc021fb1ea65 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:24.257411 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48587b-9ad5-490a-a704-8f43f7ed4d2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:24.258886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48587b-9ad5-490a-a704-8f43f7ed4d2a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:24.328252 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76f916-d0ef-4709-a4f2-bc021fb1ea65 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:24.332936 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:24.367017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8d055-f172-45e2-9d21-900bc8c07db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:24.368098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8d055-f172-45e2-9d21-900bc8c07db0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:24.368603 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8d055-f172-45e2-9d21-900bc8c07db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:24.369093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8d055-f172-45e2-9d21-900bc8c07db0 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:24.369998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8d055-f172-45e2-9d21-900bc8c07db0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:24.370495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8d055-f172-45e2-9d21-900bc8c07db0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:24.371449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 454/903] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:22:23 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:22:24.372303 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8d055-f172-45e2-9d21-900bc8c07db0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:24.378987 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48587b-9ad5-490a-a704-8f43f7ed4d2a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:24.382141 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:24.387458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 450/904] 10.222.0.64 () {56 vars in 1126 bytes} [Mon May 23 15:22:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:24.393327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8d055-f172-45e2-9d21-900bc8c07db0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:24.483987 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48587b-9ad5-490a-a704-8f43f7ed4d2a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:24.503733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8d055-f172-45e2-9d21-900bc8c07db0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:24.519522 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e19523-999e-4eae-9a14-ee17202da19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:24.520325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e19523-999e-4eae-9a14-ee17202da19c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:24.520613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e19523-999e-4eae-9a14-ee17202da19c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:24.520895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e19523-999e-4eae-9a14-ee17202da19c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:24.531925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e19523-999e-4eae-9a14-ee17202da19c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:24.533914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e19523-999e-4eae-9a14-ee17202da19c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:24.543372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a350cf-3f90-492b-9b9d-a9378b574abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:24.544351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a350cf-3f90-492b-9b9d-a9378b574abd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:24.544847 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a350cf-3f90-492b-9b9d-a9378b574abd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:24.545411 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a350cf-3f90-492b-9b9d-a9378b574abd None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:24.546330 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a350cf-3f90-492b-9b9d-a9378b574abd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:24.546863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a350cf-3f90-492b-9b9d-a9378b574abd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:24.549010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a350cf-3f90-492b-9b9d-a9378b574abd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:24.553023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-a1e19523-999e-4eae-9a14-ee17202da19c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:24.553023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:24.553023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:24.553023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:24.553023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:24.553023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:24.553023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:24.553023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:24.553023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:24.553023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:24.553023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:24.553023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:24.553023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:24.553023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:24.553023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:24.554262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:24.555188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:24.555188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:24.555188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:24.555188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:24.562176 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 455/906] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:24.562176 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e19523-999e-4eae-9a14-ee17202da19c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:24.562562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 451/906] 10.222.0.64 () {56 vars in 1126 bytes} [Mon May 23 15:22:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:24.563741 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a350cf-3f90-492b-9b9d-a9378b574abd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:24.585453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:24.629461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:24.666373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e19523-999e-4eae-9a14-ee17202da19c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:24.670799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a350cf-3f90-492b-9b9d-a9378b574abd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:24.705811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a3f39-8789-45c3-888e-19fa855e700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:24.706494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a3f39-8789-45c3-888e-19fa855e700a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:24.706793 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a3f39-8789-45c3-888e-19fa855e700a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:24.707066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a3f39-8789-45c3-888e-19fa855e700a None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:24.707886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a3f39-8789-45c3-888e-19fa855e700a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:24.708100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a3f39-8789-45c3-888e-19fa855e700a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:24.709657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a3f39-8789-45c3-888e-19fa855e700a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:24.724678 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722ca5f-17b8-4d34-8d87-8aa06e4e41e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:24.725745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722ca5f-17b8-4d34-8d87-8aa06e4e41e9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:24.726243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722ca5f-17b8-4d34-8d87-8aa06e4e41e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:24.726731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722ca5f-17b8-4d34-8d87-8aa06e4e41e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:24.727598 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 456/907] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:24.727970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722ca5f-17b8-4d34-8d87-8aa06e4e41e9 None admin] RBAC: Adding query filter params (interface=public, service_id=56b103fd7d50431b90537e4717a79d18, region_id=RegionOne) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:24.728505 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722ca5f-17b8-4d34-8d87-8aa06e4e41e9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:24.728959 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a3f39-8789-45c3-888e-19fa855e700a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:24.730331 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722ca5f-17b8-4d34-8d87-8aa06e4e41e9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:24.740651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 452/908] 10.222.0.64 () {56 vars in 1251 bytes} [Mon May 23 15:22:24 2022] GET /identity/v3/endpoints?service_id=56b103fd7d50431b90537e4717a79d18&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:24.746130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722ca5f-17b8-4d34-8d87-8aa06e4e41e9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:24.822303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:24.843487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a3f39-8789-45c3-888e-19fa855e700a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:24.851929 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722ca5f-17b8-4d34-8d87-8aa06e4e41e9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:24.880260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:24.884001 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3af52c-f8b6-41e3-b962-d4b836eb1879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:24.884971 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3af52c-f8b6-41e3-b962-d4b836eb1879 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:24.885456 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3af52c-f8b6-41e3-b962-d4b836eb1879 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:24.885964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3af52c-f8b6-41e3-b962-d4b836eb1879 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:24.889486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191467bb-a451-4bf3-9738-3aca332d5966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:24.890348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191467bb-a451-4bf3-9738-3aca332d5966 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:24.890807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191467bb-a451-4bf3-9738-3aca332d5966 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:24.895814 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191467bb-a451-4bf3-9738-3aca332d5966 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:24.895814 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191467bb-a451-4bf3-9738-3aca332d5966 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:24.896003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3af52c-f8b6-41e3-b962-d4b836eb1879 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:24.896110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191467bb-a451-4bf3-9738-3aca332d5966 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:24.896188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3af52c-f8b6-41e3-b962-d4b836eb1879 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:24.900657 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 453/909] 10.222.0.64 () {56 vars in 1087 bytes} [Mon May 23 15:22:24 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:22:24.915588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-9e3af52c-f8b6-41e3-b962-d4b836eb1879 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:24.915588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:24.915588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:24.915588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:24.915588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:24.915588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:24.915588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:24.915588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:24.915588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:24.915588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:24.915588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:24.915588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:24.915588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:24.915588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:24.915588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:24.916584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:24.917595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:24.917595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:24.917595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:24.917595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:24.917595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:24.917595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:24.917595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:24.917595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:24.920474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 457/910] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:24.926249 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191467bb-a451-4bf3-9738-3aca332d5966 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:25.042986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191467bb-a451-4bf3-9738-3aca332d5966 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:25.082263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e510b1-a026-4a7b-818d-074e9eecc15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:25.083462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e510b1-a026-4a7b-818d-074e9eecc15e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:25.083960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e510b1-a026-4a7b-818d-074e9eecc15e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:25.084446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e510b1-a026-4a7b-818d-074e9eecc15e None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:25.085518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e510b1-a026-4a7b-818d-074e9eecc15e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:25.086051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e510b1-a026-4a7b-818d-074e9eecc15e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:25.087754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e510b1-a026-4a7b-818d-074e9eecc15e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:25.102168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 454/911] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:25.108757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3af52c-f8b6-41e3-b962-d4b836eb1879 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:25.225096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3af52c-f8b6-41e3-b962-d4b836eb1879 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:25.261016 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1a015-7e53-4df6-8913-b941edad0351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:25.261714 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1a015-7e53-4df6-8913-b941edad0351 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:25.262004 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1a015-7e53-4df6-8913-b941edad0351 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:25.262293 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1a015-7e53-4df6-8913-b941edad0351 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:25.263129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1a015-7e53-4df6-8913-b941edad0351 None admin] RBAC: Adding query filter params (role.id=91ddccb742f94d19958d422cab370fa3, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=08dad0888c8542f4aed748a1a35db2c4) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:25.263449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1a015-7e53-4df6-8913-b941edad0351 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:25.264963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1a015-7e53-4df6-8913-b941edad0351 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:25.271656 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 458/912] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:22:25 2022] GET /identity/v3/role_assignments?user.id=08dad0888c8542f4aed748a1a35db2c4&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=91ddccb742f94d19958d422cab370fa3 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:26.099619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5778a90-63e3-456a-8c77-a85036ac3172 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:26.100160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5778a90-63e3-456a-8c77-a85036ac3172 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:26.100160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5778a90-63e3-456a-8c77-a85036ac3172 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:26.101896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 455/913] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:26.107980 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde77462-e472-45b5-aa54-ac8a6faa5064 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:26.108502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde77462-e472-45b5-aa54-ac8a6faa5064 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:26.108580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde77462-e472-45b5-aa54-ac8a6faa5064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:26.116440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:26.127684 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-cde77462-e472-45b5-aa54-ac8a6faa5064 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:26.128880 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cde77462-e472-45b5-aa54-ac8a6faa5064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:26.198810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 459/914] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:26.220775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5778a90-63e3-456a-8c77-a85036ac3172 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:26.286086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:26.306453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0412f3-dace-48e7-8797-6a58398cd614 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:26.306765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0412f3-dace-48e7-8797-6a58398cd614 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:26.307037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0412f3-dace-48e7-8797-6a58398cd614 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:26.308528 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 460/915] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:26.316296 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b50477-74ca-44b1-856d-8fa82e969bcc None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:26.317012 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b50477-74ca-44b1-856d-8fa82e969bcc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:26.317577 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b50477-74ca-44b1-856d-8fa82e969bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:26.338184 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5778a90-63e3-456a-8c77-a85036ac3172 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:26.339386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d6b50477-74ca-44b1-856d-8fa82e969bcc None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:26.345607 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d6b50477-74ca-44b1-856d-8fa82e969bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:26.382834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d6b65-7af3-466f-8860-24b5ad57fde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:26.383584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d6b65-7af3-466f-8860-24b5ad57fde4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:26.383874 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d6b65-7af3-466f-8860-24b5ad57fde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:26.384153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d6b65-7af3-466f-8860-24b5ad57fde4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:26.384824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d6b65-7af3-466f-8860-24b5ad57fde4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:26.386251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d6b65-7af3-466f-8860-24b5ad57fde4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-261d6b65-7af3-466f-8860-24b5ad57fde4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:26.399538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:26.400570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:26.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:26.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:26.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:26.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:26.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:26.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:26.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:26.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:26.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:26.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:26.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:26.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:26.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:22:26.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:26.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 456/916] 10.222.0.64 () {56 vars in 1126 bytes} [Mon May 23 15:22:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:26.406607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d6b65-7af3-466f-8860-24b5ad57fde4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:26.439450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 461/917] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:26.462264 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b50477-74ca-44b1-856d-8fa82e969bcc None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:26.523885 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d6b65-7af3-466f-8860-24b5ad57fde4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:26.547563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:26.555415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:26.573698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4db76-9a67-41c6-8dce-68ba52bad2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:26.574945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4db76-9a67-41c6-8dce-68ba52bad2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:26.575462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4db76-9a67-41c6-8dce-68ba52bad2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:26.576083 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4db76-9a67-41c6-8dce-68ba52bad2a6 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:26.577527 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4db76-9a67-41c6-8dce-68ba52bad2a6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:26.578215 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4db76-9a67-41c6-8dce-68ba52bad2a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:26.580936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4db76-9a67-41c6-8dce-68ba52bad2a6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:26.587653 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 457/918] 10.222.0.64 () {56 vars in 1122 bytes} [Mon May 23 15:22:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:26.593281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4db76-9a67-41c6-8dce-68ba52bad2a6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:26.604013 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b50477-74ca-44b1-856d-8fa82e969bcc None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:26.655700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ad125-1800-487e-b885-c51e3d078300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:26.657101 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ad125-1800-487e-b885-c51e3d078300 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:26.658017 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ad125-1800-487e-b885-c51e3d078300 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:26.658915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ad125-1800-487e-b885-c51e3d078300 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:26.660056 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ad125-1800-487e-b885-c51e3d078300 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:26.662592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ad125-1800-487e-b885-c51e3d078300 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c04ad125-1800-487e-b885-c51e3d078300 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:26.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:26.680804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:26.681699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:26.681699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:26.681699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:26.681699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:26.681699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:26.681699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:26.681699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:26.681699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:26.681699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:26.681699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:26.681699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:26.681699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:26.681699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:22:26.681699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:26.683116 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 462/919] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:22:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:26.688117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ad125-1800-487e-b885-c51e3d078300 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:26.708674 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4db76-9a67-41c6-8dce-68ba52bad2a6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:26.748131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b19162-12c2-4dc7-b472-e8ce1e22ab91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:26.749634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b19162-12c2-4dc7-b472-e8ce1e22ab91 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:26.750415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b19162-12c2-4dc7-b472-e8ce1e22ab91 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:26.750914 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b19162-12c2-4dc7-b472-e8ce1e22ab91 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:26.752638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b19162-12c2-4dc7-b472-e8ce1e22ab91 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:26.753657 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b19162-12c2-4dc7-b472-e8ce1e22ab91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:26.755790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b19162-12c2-4dc7-b472-e8ce1e22ab91 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:26.763401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 458/920] 10.222.0.64 () {56 vars in 1122 bytes} [Mon May 23 15:22:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:26.769129 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b19162-12c2-4dc7-b472-e8ce1e22ab91 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:26.811223 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ad125-1800-487e-b885-c51e3d078300 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:26.835695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:26.840340 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:26.848756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734b396-5bd5-4bdb-8088-7eeaf6b2f10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:26.849580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734b396-5bd5-4bdb-8088-7eeaf6b2f10f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:26.849802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734b396-5bd5-4bdb-8088-7eeaf6b2f10f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:26.850141 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734b396-5bd5-4bdb-8088-7eeaf6b2f10f None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:26.850861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734b396-5bd5-4bdb-8088-7eeaf6b2f10f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:26.851178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734b396-5bd5-4bdb-8088-7eeaf6b2f10f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:26.852784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734b396-5bd5-4bdb-8088-7eeaf6b2f10f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:26.861382 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 463/921] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:22:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:26.864352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734b396-5bd5-4bdb-8088-7eeaf6b2f10f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:26.908169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b19162-12c2-4dc7-b472-e8ce1e22ab91 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:26.952699 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714af85-8d4a-443e-88cb-b4890e6c1b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:26.953927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714af85-8d4a-443e-88cb-b4890e6c1b2a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:26.955106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714af85-8d4a-443e-88cb-b4890e6c1b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:26.956123 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714af85-8d4a-443e-88cb-b4890e6c1b2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:26.957464 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714af85-8d4a-443e-88cb-b4890e6c1b2a None admin] RBAC: Adding query filter params (interface=public, service_id=c25a7fa74aeb428bb823ff97514e3bb2, region_id=RegionOne) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:26.958021 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714af85-8d4a-443e-88cb-b4890e6c1b2a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:26.959981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714af85-8d4a-443e-88cb-b4890e6c1b2a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:26.967460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 459/922] 10.222.0.64 () {56 vars in 1251 bytes} [Mon May 23 15:22:26 2022] GET /identity/v3/endpoints?service_id=c25a7fa74aeb428bb823ff97514e3bb2&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:26.972658 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714af85-8d4a-443e-88cb-b4890e6c1b2a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:26.978832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734b396-5bd5-4bdb-8088-7eeaf6b2f10f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:27.013641 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a72db3-7228-4dc7-a701-2c6d3e53e92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:27.014331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a72db3-7228-4dc7-a701-2c6d3e53e92c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.014642 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a72db3-7228-4dc7-a701-2c6d3e53e92c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.014952 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a72db3-7228-4dc7-a701-2c6d3e53e92c None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.015609 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a72db3-7228-4dc7-a701-2c6d3e53e92c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:27.015920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a72db3-7228-4dc7-a701-2c6d3e53e92c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:27.017872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a72db3-7228-4dc7-a701-2c6d3e53e92c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:27.023991 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 464/923] 10.222.0.64 () {56 vars in 1114 bytes} [Mon May 23 15:22:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:27.029508 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a72db3-7228-4dc7-a701-2c6d3e53e92c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:27.097813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714af85-8d4a-443e-88cb-b4890e6c1b2a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:27.107495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:27.113423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:27.135778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2373e08-bedf-4f47-96f5-3485bb8fa59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:27.136492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2373e08-bedf-4f47-96f5-3485bb8fa59c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.136786 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2373e08-bedf-4f47-96f5-3485bb8fa59c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.137082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2373e08-bedf-4f47-96f5-3485bb8fa59c None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.137829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2373e08-bedf-4f47-96f5-3485bb8fa59c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:27.139205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2373e08-bedf-4f47-96f5-3485bb8fa59c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:27.152997 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 460/924] 10.222.0.64 () {56 vars in 1087 bytes} [Mon May 23 15:22:26 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:22:27.170184 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a72db3-7228-4dc7-a701-2c6d3e53e92c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:27.210139 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dba2e0-a6a2-4300-a476-5735454ce828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:27.211276 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dba2e0-a6a2-4300-a476-5735454ce828 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.211781 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dba2e0-a6a2-4300-a476-5735454ce828 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.212273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dba2e0-a6a2-4300-a476-5735454ce828 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.213310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dba2e0-a6a2-4300-a476-5735454ce828 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:27.215054 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dba2e0-a6a2-4300-a476-5735454ce828 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:27.266130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c22e8b-10ed-4845-bae3-bc89ca90ffb0 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.266647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c22e8b-10ed-4845-bae3-bc89ca90ffb0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.267476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c22e8b-10ed-4845-bae3-bc89ca90ffb0 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.269569 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 461/925] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:27.275073 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6072136-0dbe-4368-80ce-a0abfd29a869 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.275435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6072136-0dbe-4368-80ce-a0abfd29a869 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.275657 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6072136-0dbe-4368-80ce-a0abfd29a869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.294953 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b6072136-0dbe-4368-80ce-a0abfd29a869 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:27.296282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 465/926] 10.222.0.64 () {60 vars in 1144 bytes} [Mon May 23 15:22:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:22:27.297413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b6072136-0dbe-4368-80ce-a0abfd29a869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:27.396326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 462/927] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:27.419433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dba2e0-a6a2-4300-a476-5735454ce828 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:27.512215 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a11e5-a359-4d14-8069-db91164e5dfc None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.512458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a11e5-a359-4d14-8069-db91164e5dfc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.512882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a11e5-a359-4d14-8069-db91164e5dfc None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.514490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 463/928] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:27.522378 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9b43c-6e8a-4639-9f2a-bac926745642 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.522924 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9b43c-6e8a-4639-9f2a-bac926745642 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.523795 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9b43c-6e8a-4639-9f2a-bac926745642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.538529 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dba2e0-a6a2-4300-a476-5735454ce828 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:27.546485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a0e9b43c-6e8a-4639-9f2a-bac926745642 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:27.547692 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a0e9b43c-6e8a-4639-9f2a-bac926745642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:27.554251 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:27.581914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe2b73-92f8-4fbd-b3e7-a156c4201ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:27.582981 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe2b73-92f8-4fbd-b3e7-a156c4201ccf None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.583267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe2b73-92f8-4fbd-b3e7-a156c4201ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.583534 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe2b73-92f8-4fbd-b3e7-a156c4201ccf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.584198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe2b73-92f8-4fbd-b3e7-a156c4201ccf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:27.585666 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe2b73-92f8-4fbd-b3e7-a156c4201ccf None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c5fe2b73-92f8-4fbd-b3e7-a156c4201ccf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:27.599417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:27.600766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:27.601771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 466/929] 10.222.0.64 () {56 vars in 1132 bytes} [Mon May 23 15:22:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:27.606634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe2b73-92f8-4fbd-b3e7-a156c4201ccf None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:27.637117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 464/930] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:27.660010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9b43c-6e8a-4639-9f2a-bac926745642 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:27.690729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:27.720640 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe2b73-92f8-4fbd-b3e7-a156c4201ccf None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:27.757788 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe24777-f4c7-4838-b3c6-fce90514a1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:27.759291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe24777-f4c7-4838-b3c6-fce90514a1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.759936 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe24777-f4c7-4838-b3c6-fce90514a1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.760497 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe24777-f4c7-4838-b3c6-fce90514a1d5 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.761581 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe24777-f4c7-4838-b3c6-fce90514a1d5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:27.762093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe24777-f4c7-4838-b3c6-fce90514a1d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:27.764225 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe24777-f4c7-4838-b3c6-fce90514a1d5 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:27.775616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9b43c-6e8a-4639-9f2a-bac926745642 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:27.775807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 467/931] 10.222.0.64 () {56 vars in 1126 bytes} [Mon May 23 15:22:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:27.776341 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe24777-f4c7-4838-b3c6-fce90514a1d5 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:27.813255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6748ae-3052-4141-8db0-4e9f72d51e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:27.815047 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6748ae-3052-4141-8db0-4e9f72d51e23 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.815341 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6748ae-3052-4141-8db0-4e9f72d51e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.815621 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6748ae-3052-4141-8db0-4e9f72d51e23 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.827373 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6748ae-3052-4141-8db0-4e9f72d51e23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:27.833935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-0d6748ae-3052-4141-8db0-4e9f72d51e23 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:27.836763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:27.837762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:27.838649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:27.838649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:27.838649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:27.838649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:27.838649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:27.838649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:27.838649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:27.838649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:27.838649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:27.838649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:27.838649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:22:27.838649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:27.838649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 465/932] 10.222.0.64 () {56 vars in 1120 bytes} [Mon May 23 15:22:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:27.845764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6748ae-3052-4141-8db0-4e9f72d51e23 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:27.910117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe24777-f4c7-4838-b3c6-fce90514a1d5 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:27.948173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:27.956835 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18527e77-44c4-4a44-b354-9f93bda2c232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:27.957534 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18527e77-44c4-4a44-b354-9f93bda2c232 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:27.957908 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18527e77-44c4-4a44-b354-9f93bda2c232 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:27.958127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18527e77-44c4-4a44-b354-9f93bda2c232 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:27.958789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18527e77-44c4-4a44-b354-9f93bda2c232 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:27.959095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18527e77-44c4-4a44-b354-9f93bda2c232 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:27.960488 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18527e77-44c4-4a44-b354-9f93bda2c232 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:27.966865 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6748ae-3052-4141-8db0-4e9f72d51e23 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:27.974252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 468/933] 10.222.0.64 () {56 vars in 1126 bytes} [Mon May 23 15:22:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:27.975487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18527e77-44c4-4a44-b354-9f93bda2c232 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:28.008117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6d1b5-e278-484c-a103-56939056481c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:28.009043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6d1b5-e278-484c-a103-56939056481c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:28.009509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6d1b5-e278-484c-a103-56939056481c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:28.010049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6d1b5-e278-484c-a103-56939056481c None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:28.011009 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6d1b5-e278-484c-a103-56939056481c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:28.011326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6d1b5-e278-484c-a103-56939056481c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:28.012752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6d1b5-e278-484c-a103-56939056481c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:28.027137 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 466/934] 10.222.0.64 () {56 vars in 1115 bytes} [Mon May 23 15:22:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:28.028402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6d1b5-e278-484c-a103-56939056481c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:28.118774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18527e77-44c4-4a44-b354-9f93bda2c232 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:28.122481 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:28.137872 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6d1b5-e278-484c-a103-56939056481c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:28.166267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03953733-4626-4386-8fd1-c245ae37263d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:28.167725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03953733-4626-4386-8fd1-c245ae37263d None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:28.168300 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03953733-4626-4386-8fd1-c245ae37263d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:28.168764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03953733-4626-4386-8fd1-c245ae37263d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:28.169880 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03953733-4626-4386-8fd1-c245ae37263d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:28.172164 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03953733-4626-4386-8fd1-c245ae37263d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:28.175121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c316f5-bead-4814-8d62-c0c4f6b057a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:28.176150 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c316f5-bead-4814-8d62-c0c4f6b057a2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:28.176731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c316f5-bead-4814-8d62-c0c4f6b057a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:28.177107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c316f5-bead-4814-8d62-c0c4f6b057a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:28.189430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c316f5-bead-4814-8d62-c0c4f6b057a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:28.190984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c316f5-bead-4814-8d62-c0c4f6b057a2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:28.209813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-47c316f5-bead-4814-8d62-c0c4f6b057a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:28.209813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:28.209813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:28.209813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:28.209813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:28.209813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:28.209813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:28.209813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:28.209813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:28.209813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:28.209813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:28.209813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:28.209813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:28.209813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:28.209813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:28.210767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:28.211780 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:28.211780 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:28.211780 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:28.211780 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:28.211780 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 467/935] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:28.217519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c316f5-bead-4814-8d62-c0c4f6b057a2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:28.220498 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:28.293086 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 469/936] 10.222.0.64 () {60 vars in 1144 bytes} [Mon May 23 15:22:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:22:28.334830 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c316f5-bead-4814-8d62-c0c4f6b057a2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:28.382199 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d08ff8-14c6-4e21-90bb-8b94efe6a032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:28.383024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d08ff8-14c6-4e21-90bb-8b94efe6a032 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:28.383375 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d08ff8-14c6-4e21-90bb-8b94efe6a032 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:28.383719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d08ff8-14c6-4e21-90bb-8b94efe6a032 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:28.384566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d08ff8-14c6-4e21-90bb-8b94efe6a032 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:28.384996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d08ff8-14c6-4e21-90bb-8b94efe6a032 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:28.386523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d08ff8-14c6-4e21-90bb-8b94efe6a032 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:28.400469 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 468/937] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:28.406089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03953733-4626-4386-8fd1-c245ae37263d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:28.519766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03953733-4626-4386-8fd1-c245ae37263d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:28.555530 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881c034-03dc-46ea-8180-d13050090b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:28.556402 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881c034-03dc-46ea-8180-d13050090b4b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:28.556751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881c034-03dc-46ea-8180-d13050090b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:28.557118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881c034-03dc-46ea-8180-d13050090b4b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:28.565096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881c034-03dc-46ea-8180-d13050090b4b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:28.566637 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881c034-03dc-46ea-8180-d13050090b4b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-7881c034-03dc-46ea-8180-d13050090b4b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:22:28.582311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:28.583556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:28.584766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:28.584766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:28.584766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:22:28.584766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:22:28.584766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:22:28.584766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:22:28.584766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:22:28.584766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:22:28.584766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:22:28.584766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:28.585709 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 470/938] 10.222.0.64 () {56 vars in 1117 bytes} [Mon May 23 15:22:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:28.591744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d08ff8-14c6-4e21-90bb-8b94efe6a032 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:28.656639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:28.714201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d08ff8-14c6-4e21-90bb-8b94efe6a032 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:28.751420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990f240-7395-4c0a-ac74-94cee9ab003f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:28.752197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990f240-7395-4c0a-ac74-94cee9ab003f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:28.752494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990f240-7395-4c0a-ac74-94cee9ab003f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:28.752779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990f240-7395-4c0a-ac74-94cee9ab003f None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:28.753666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990f240-7395-4c0a-ac74-94cee9ab003f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:28.753985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990f240-7395-4c0a-ac74-94cee9ab003f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:28.755424 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990f240-7395-4c0a-ac74-94cee9ab003f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:28.774777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 469/939] 10.222.0.64 () {56 vars in 1149 bytes} [Mon May 23 15:22:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:28.780233 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881c034-03dc-46ea-8180-d13050090b4b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:28.805116 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:28.884572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881c034-03dc-46ea-8180-d13050090b4b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:28.920552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74677a-aea3-4d22-849b-089a8e9e5cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:28.921429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74677a-aea3-4d22-849b-089a8e9e5cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:28.921726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74677a-aea3-4d22-849b-089a8e9e5cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:28.922006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74677a-aea3-4d22-849b-089a8e9e5cd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:28.933477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74677a-aea3-4d22-849b-089a8e9e5cd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:28.934962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74677a-aea3-4d22-849b-089a8e9e5cd2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:28.953274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-6b74677a-aea3-4d22-849b-089a8e9e5cd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:28.953274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:28.953274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:28.953274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:28.953274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:28.953274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:28.953274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:28.953274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:28.953274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:28.953274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:28.953274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:28.953274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:28.953274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:28.953274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:28.953274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:28.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:28.955170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:28.955170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:28.955170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:28.955170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:28.955170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 471/940] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:28.961451 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990f240-7395-4c0a-ac74-94cee9ab003f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:29.068461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990f240-7395-4c0a-ac74-94cee9ab003f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:29.086868 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:29.105845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73090a13-99c0-4b9e-a6c0-a1bb68b9722b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:29.111029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73090a13-99c0-4b9e-a6c0-a1bb68b9722b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:29.111029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73090a13-99c0-4b9e-a6c0-a1bb68b9722b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:29.111029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73090a13-99c0-4b9e-a6c0-a1bb68b9722b None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:29.111029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73090a13-99c0-4b9e-a6c0-a1bb68b9722b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:29.111029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73090a13-99c0-4b9e-a6c0-a1bb68b9722b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:29.111029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73090a13-99c0-4b9e-a6c0-a1bb68b9722b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:29.122416 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 470/941] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:29.128613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74677a-aea3-4d22-849b-089a8e9e5cd2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:29.205164 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:29.233019 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74677a-aea3-4d22-849b-089a8e9e5cd2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:29.268956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142536ca-34df-446c-b196-005adf7a6c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:29.269800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142536ca-34df-446c-b196-005adf7a6c07 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:29.270086 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142536ca-34df-446c-b196-005adf7a6c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:29.270369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142536ca-34df-446c-b196-005adf7a6c07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:29.277469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142536ca-34df-446c-b196-005adf7a6c07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:29.279015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142536ca-34df-446c-b196-005adf7a6c07 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:29.294311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-142536ca-34df-446c-b196-005adf7a6c07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:29.294311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:29.294311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:29.294311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:29.294311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:29.294311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:29.294311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:29.294311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:29.294311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:29.294311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:29.294311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:29.294311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:29.294311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:29.294311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:29.294311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:29.295294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:29.296240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:29.296240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:29.296240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:29.296240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:29.296240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:29.296240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:29.296240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:29.296240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:29.296240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 472/942] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:29.304533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73090a13-99c0-4b9e-a6c0-a1bb68b9722b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:29.416006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73090a13-99c0-4b9e-a6c0-a1bb68b9722b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:29.450967 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23807cc-35de-476f-bb27-a6c5d118d116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:29.451637 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23807cc-35de-476f-bb27-a6c5d118d116 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:29.451928 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23807cc-35de-476f-bb27-a6c5d118d116 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:29.452254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23807cc-35de-476f-bb27-a6c5d118d116 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:29.453008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23807cc-35de-476f-bb27-a6c5d118d116 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:29.453352 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23807cc-35de-476f-bb27-a6c5d118d116 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:29.455315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23807cc-35de-476f-bb27-a6c5d118d116 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:29.468298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 471/943] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:29.474713 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142536ca-34df-446c-b196-005adf7a6c07 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:29.491661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa25a19-1078-473d-a11f-eebee78d9cb8 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:29.491948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa25a19-1078-473d-a11f-eebee78d9cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:29.492237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa25a19-1078-473d-a11f-eebee78d9cb8 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:29.494909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 472/944] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:29.498847 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85a2c7-ad07-457a-b6d7-ee3c4514eb9a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:29.499143 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85a2c7-ad07-457a-b6d7-ee3c4514eb9a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:29.499429 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85a2c7-ad07-457a-b6d7-ee3c4514eb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:29.516580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1b85a2c7-ad07-457a-b6d7-ee3c4514eb9a None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:29.517805 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1b85a2c7-ad07-457a-b6d7-ee3c4514eb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:29.550559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:29.600448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142536ca-34df-446c-b196-005adf7a6c07 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:29.607823 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 473/945] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:29.632943 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85a2c7-ad07-457a-b6d7-ee3c4514eb9a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:29.649168 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:29.662037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56d9ca-1f6c-42f1-8438-9c6ae911d241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:29.663507 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56d9ca-1f6c-42f1-8438-9c6ae911d241 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:29.664401 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56d9ca-1f6c-42f1-8438-9c6ae911d241 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:29.664834 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56d9ca-1f6c-42f1-8438-9c6ae911d241 None admin] PATH_INFO: `/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/08dad0888c8542f4aed748a1a35db2c4/roles/91ddccb742f94d19958d422cab370fa3` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:29.668670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56d9ca-1f6c-42f1-8438-9c6ae911d241 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d81eafac474f37a71ee5cc292d172f, user_id=08dad0888c8542f4aed748a1a35db2c4, role_id=91ddccb742f94d19958d422cab370fa3)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:29.671259 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56d9ca-1f6c-42f1-8438-9c6ae911d241 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:29.750611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 473/946] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:22:29 2022] PUT /identity/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/08dad0888c8542f4aed748a1a35db2c4/roles/91ddccb742f94d19958d422cab370fa3 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:22:29.755609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85a2c7-ad07-457a-b6d7-ee3c4514eb9a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:29.775700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6187f-7e76-45d3-835d-5f3889a9064f None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:29.776051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6187f-7e76-45d3-835d-5f3889a9064f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:29.776346 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6187f-7e76-45d3-835d-5f3889a9064f None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:29.781893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 474/947] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:29.784916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27924cba-eba9-4703-bfe0-019762b5760d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:29.785538 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27924cba-eba9-4703-bfe0-019762b5760d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:29.786796 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27924cba-eba9-4703-bfe0-019762b5760d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:29.804121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb03fb-d8b4-437a-af86-4ec86db44364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:29.804815 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb03fb-d8b4-437a-af86-4ec86db44364 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:29.805132 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb03fb-d8b4-437a-af86-4ec86db44364 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:29.805472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb03fb-d8b4-437a-af86-4ec86db44364 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:29.806157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb03fb-d8b4-437a-af86-4ec86db44364 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:29.807671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb03fb-d8b4-437a-af86-4ec86db44364 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:29.813983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-27924cba-eba9-4703-bfe0-019762b5760d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:29.815699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-27924cba-eba9-4703-bfe0-019762b5760d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6fbb03fb-d8b4-437a-af86-4ec86db44364 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:29.822270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:29.823512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:29.825982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:29.825982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:29.825982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:29.825982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:29.825982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:29.825982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:29.825982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:29.825982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:29.825982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:29.825982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:29.825982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:29.825982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:29.825982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:22:29.825982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:29.825982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 474/948] 10.222.0.64 () {56 vars in 1126 bytes} [Mon May 23 15:22:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:29.831208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb03fb-d8b4-437a-af86-4ec86db44364 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:29.869023 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:29.925225 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 475/949] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:29.952423 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb03fb-d8b4-437a-af86-4ec86db44364 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:29.953114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27924cba-eba9-4703-bfe0-019762b5760d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:30.003160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239830a-4e18-46f9-acee-8d0602fcc53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:30.003759 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239830a-4e18-46f9-acee-8d0602fcc53e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:30.004043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239830a-4e18-46f9-acee-8d0602fcc53e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:30.004317 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239830a-4e18-46f9-acee-8d0602fcc53e None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:30.005171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239830a-4e18-46f9-acee-8d0602fcc53e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:30.005482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239830a-4e18-46f9-acee-8d0602fcc53e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:30.006956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239830a-4e18-46f9-acee-8d0602fcc53e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:30.014558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 475/950] 10.222.0.64 () {56 vars in 1122 bytes} [Mon May 23 15:22:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:30.021118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239830a-4e18-46f9-acee-8d0602fcc53e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:30.073017 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27924cba-eba9-4703-bfe0-019762b5760d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:30.104049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:30.120141 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c819a-b550-49ab-8a19-49f359042ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:30.121374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c819a-b550-49ab-8a19-49f359042ff4 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:30.122156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c819a-b550-49ab-8a19-49f359042ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:30.122910 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c819a-b550-49ab-8a19-49f359042ff4 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:30.124078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c819a-b550-49ab-8a19-49f359042ff4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:30.125509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:30.127573 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c819a-b550-49ab-8a19-49f359042ff4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:30.129214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: WARNING keystone.server.flask.common [None req-402c819a-b550-49ab-8a19-49f359042ff4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 23 15:22:30.129762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239830a-4e18-46f9-acee-8d0602fcc53e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:30.166977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea0434-5f39-489d-ae63-642bcb3bd3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:30.168336 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea0434-5f39-489d-ae63-642bcb3bd3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:30.169166 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea0434-5f39-489d-ae63-642bcb3bd3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:30.169448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea0434-5f39-489d-ae63-642bcb3bd3c1 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:30.170767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea0434-5f39-489d-ae63-642bcb3bd3c1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:30.171200 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea0434-5f39-489d-ae63-642bcb3bd3c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:30.173387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea0434-5f39-489d-ae63-642bcb3bd3c1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:30.183797 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 476/951] 10.222.0.64 () {56 vars in 1122 bytes} [Mon May 23 15:22:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:30.184944 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea0434-5f39-489d-ae63-642bcb3bd3c1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:30.190294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.sql.core [None req-402c819a-b550-49ab-8a19-49f359042ff4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 15:22:30.190294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 15:22:30.190294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [parameters: {'id': '399ae89bd2464aab9c21ffbf79c6cb4c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 15:22:30.190294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80598) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 15:22:30.207692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-402c819a-b550-49ab-8a19-49f359042ff4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 399ae89bd2464aab9c21ffbf79c6cb4c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 399ae89bd2464aab9c21ffbf79c6cb4c. May 23 15:22:30.207692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:30.207692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:22:30.207692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:22:30.207692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:22:30.207692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:22:30.207692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:22:30.207692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:22:30.207692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:22:30.207692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application conn.query(q) May 23 15:22:30.207692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:22:30.207692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:22:30.207692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result.read() May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 23 15:22:30.208597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return project_ref.to_dict() May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application next(self.gen) May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application yield resource May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application next(self.gen) May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.session.rollback() May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.force_reraise() May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise self.value May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application session.commit() May 23 15:22:30.209649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._prepare_impl() May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.session.flush() May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._flush(objects) May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application compat.raise_( May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application flush_context.execute() May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 15:22:30.210632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rec.execute(self) May 23 15:22:30.212403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 15:22:30.212403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 15:22:30.212403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 15:22:30.212403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 15:22:30.212403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 15:22:30.212403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 15:22:30.212403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 15:22:30.212403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 15:22:30.212403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 15:22:30.212403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 15:22:30.212403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 15:22:30.212403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 15:22:30.212403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 15:22:30.212403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 15:22:30.212403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application conn.query(q) May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application result.read() May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:22:30.214688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:22:30.217032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:22:30.217032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:22:30.217032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:22:30.217032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:22:30.217032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:22:30.217032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:22:30.217032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 15:22:30.217032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 15:22:30.217032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [parameters: {'id': '399ae89bd2464aab9c21ffbf79c6cb4c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 15:22:30.217032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 15:22:30.217032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:30.217032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:30.217032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:30.217032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:30.217032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:30.219038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:30.223687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:30.223687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 23 15:22:30.223687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 23 15:22:30.223687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:30.223687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:30.223687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 23 15:22:30.223687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.Conflict( May 23 15:22:30.223687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 399ae89bd2464aab9c21ffbf79c6cb4c. May 23 15:22:30.223687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:30.223687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 476/952] 10.222.0.64 () {60 vars in 1140 bytes} [Mon May 23 15:22:29 2022] POST /identity/v3/projects => generated 262 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:30.223687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c819a-b550-49ab-8a19-49f359042ff4 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:30.326617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea0434-5f39-489d-ae63-642bcb3bd3c1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:30.336629 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c819a-b550-49ab-8a19-49f359042ff4 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:30.361970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616f291-f0f4-4f26-ae28-e12c40ddfeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:30.362636 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616f291-f0f4-4f26-ae28-e12c40ddfeed None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:30.362921 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616f291-f0f4-4f26-ae28-e12c40ddfeed None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:30.363198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616f291-f0f4-4f26-ae28-e12c40ddfeed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:30.363949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616f291-f0f4-4f26-ae28-e12c40ddfeed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:30.365336 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616f291-f0f4-4f26-ae28-e12c40ddfeed None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:30.374329 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f0bc3-99ff-493a-b99d-9c2068084c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:30.375194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f0bc3-99ff-493a-b99d-9c2068084c92 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:30.375572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f0bc3-99ff-493a-b99d-9c2068084c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:30.376426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f0bc3-99ff-493a-b99d-9c2068084c92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:30.383865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f0bc3-99ff-493a-b99d-9c2068084c92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:30.385862 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f0bc3-99ff-493a-b99d-9c2068084c92 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:30.400715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-643f0bc3-99ff-493a-b99d-9c2068084c92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:30.400715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:30.400715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:30.400715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:30.400715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:30.400715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:30.400715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:30.400715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:30.400715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:30.400715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:30.400715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:30.400715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:30.400715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:30.400715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:30.400715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:30.401846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:30.402787 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:30.402787 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:30.402787 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:30.402787 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:30.402787 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:30.402787 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:30.402787 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:30.402787 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:30.403758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 477/953] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:30.408088 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f0bc3-99ff-493a-b99d-9c2068084c92 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:30.436736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 477/954] 10.222.0.64 () {60 vars in 1144 bytes} [Mon May 23 15:22:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:22:30.465054 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:30.545209 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f0bc3-99ff-493a-b99d-9c2068084c92 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:30.596986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ff180-b6a7-437f-96ea-5313830cb772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:30.597851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ff180-b6a7-437f-96ea-5313830cb772 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:30.598240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ff180-b6a7-437f-96ea-5313830cb772 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:30.598624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ff180-b6a7-437f-96ea-5313830cb772 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:30.599555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ff180-b6a7-437f-96ea-5313830cb772 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:30.599972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ff180-b6a7-437f-96ea-5313830cb772 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:30.601682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ff180-b6a7-437f-96ea-5313830cb772 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:30.623375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 478/955] 10.222.0.64 () {56 vars in 1112 bytes} [Mon May 23 15:22:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:30.788951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89788551-e5b4-46e8-89ae-a7acc548c226 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:30.789277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89788551-e5b4-46e8-89ae-a7acc548c226 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:30.789571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89788551-e5b4-46e8-89ae-a7acc548c226 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:30.791406 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 478/956] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:30.797241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113f2f0-6724-4be1-a388-82826155f36c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:30.798206 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113f2f0-6724-4be1-a388-82826155f36c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:30.798591 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113f2f0-6724-4be1-a388-82826155f36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:30.816804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2113f2f0-6724-4be1-a388-82826155f36c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:30.818508 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2113f2f0-6724-4be1-a388-82826155f36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:30.889553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 479/957] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:30.913809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89788551-e5b4-46e8-89ae-a7acc548c226 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:30.931261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:31.025158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89788551-e5b4-46e8-89ae-a7acc548c226 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:31.060053 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd04a5-ac48-47d1-9433-d7cf739ab512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:31.061174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd04a5-ac48-47d1-9433-d7cf739ab512 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:31.061629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd04a5-ac48-47d1-9433-d7cf739ab512 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:31.062054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd04a5-ac48-47d1-9433-d7cf739ab512 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:31.062874 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd04a5-ac48-47d1-9433-d7cf739ab512 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:31.064930 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd04a5-ac48-47d1-9433-d7cf739ab512 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-ecfd04a5-ac48-47d1-9433-d7cf739ab512 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:31.077349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:31.078628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:31.079739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:31.079739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:31.079739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:31.079739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:31.079739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:31.079739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:31.079739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:31.079739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:31.079739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:31.079739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:31.079739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:31.079739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:31.079739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:22:31.079739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:31.082241 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 479/958] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:31.088187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113f2f0-6724-4be1-a388-82826155f36c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:31.115261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:31.194108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113f2f0-6724-4be1-a388-82826155f36c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:31.231302 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e316c-68f5-493d-883c-de58380339fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:31.232251 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e316c-68f5-493d-883c-de58380339fc None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:31.232687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e316c-68f5-493d-883c-de58380339fc None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:31.233136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e316c-68f5-493d-883c-de58380339fc None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:31.234013 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e316c-68f5-493d-883c-de58380339fc None admin] RBAC: Adding query filter params (name=compute) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:31.234477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e316c-68f5-493d-883c-de58380339fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:31.237317 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e316c-68f5-493d-883c-de58380339fc None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:31.242807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 480/959] 10.222.0.64 () {56 vars in 1112 bytes} [Mon May 23 15:22:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:31.250163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd04a5-ac48-47d1-9433-d7cf739ab512 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:31.337437 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:31.356048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd04a5-ac48-47d1-9433-d7cf739ab512 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:31.390398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c1754-31f9-456e-98de-9bd00a562ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:31.391076 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c1754-31f9-456e-98de-9bd00a562ffc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:31.391935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c1754-31f9-456e-98de-9bd00a562ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:31.392430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c1754-31f9-456e-98de-9bd00a562ffc None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:31.393345 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c1754-31f9-456e-98de-9bd00a562ffc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:31.393662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c1754-31f9-456e-98de-9bd00a562ffc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:31.395027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c1754-31f9-456e-98de-9bd00a562ffc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:31.399996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 480/960] 10.222.0.64 () {56 vars in 1112 bytes} [Mon May 23 15:22:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:32.191406 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4318ade-59fe-460c-8876-b5371c885883 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:32.192067 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4318ade-59fe-460c-8876-b5371c885883 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:32.192580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4318ade-59fe-460c-8876-b5371c885883 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:32.194484 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 481/961] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:32.200457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84073304-7f86-4dc0-a8e1-2a6045c0acf6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:32.201029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84073304-7f86-4dc0-a8e1-2a6045c0acf6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:32.201553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84073304-7f86-4dc0-a8e1-2a6045c0acf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:32.219537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-84073304-7f86-4dc0-a8e1-2a6045c0acf6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:32.221761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-84073304-7f86-4dc0-a8e1-2a6045c0acf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:32.297093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 481/962] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:32.319628 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4318ade-59fe-460c-8876-b5371c885883 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:32.418752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:32.433214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4318ade-59fe-460c-8876-b5371c885883 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:32.471338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ea329-9d03-4b23-9dd7-190395b1ed96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:32.472093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ea329-9d03-4b23-9dd7-190395b1ed96 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:32.472383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ea329-9d03-4b23-9dd7-190395b1ed96 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:32.472665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ea329-9d03-4b23-9dd7-190395b1ed96 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:32.483818 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ea329-9d03-4b23-9dd7-190395b1ed96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-272ea329-9d03-4b23-9dd7-190395b1ed96 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:32.493478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:32.494444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:32.495307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:32.495307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:32.495307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:32.495307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:32.495307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:32.495307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:32.495307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:32.495307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:32.495307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:32.495307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:32.495307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:22:32.495307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:32.499751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 482/963] 10.222.0.64 () {56 vars in 1120 bytes} [Mon May 23 15:22:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:32.502496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84073304-7f86-4dc0-a8e1-2a6045c0acf6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:32.585728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:32.616558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84073304-7f86-4dc0-a8e1-2a6045c0acf6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:32.653899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35702e21-1e1c-4b73-b42b-583b6ebdb540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:32.654648 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35702e21-1e1c-4b73-b42b-583b6ebdb540 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:32.654936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35702e21-1e1c-4b73-b42b-583b6ebdb540 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:32.655261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35702e21-1e1c-4b73-b42b-583b6ebdb540 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:32.656018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35702e21-1e1c-4b73-b42b-583b6ebdb540 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:32.656389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35702e21-1e1c-4b73-b42b-583b6ebdb540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:32.658391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35702e21-1e1c-4b73-b42b-583b6ebdb540 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:32.671550 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 482/964] 10.222.0.64 () {56 vars in 1115 bytes} [Mon May 23 15:22:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:32.673563 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ea329-9d03-4b23-9dd7-190395b1ed96 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:32.729156 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5d4d3-5300-4975-aba5-8521a83ff274 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:32.729677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5d4d3-5300-4975-aba5-8521a83ff274 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:32.730120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5d4d3-5300-4975-aba5-8521a83ff274 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:32.731974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 483/965] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:32.737362 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a82e84-8ce7-4c8b-bd45-a0a25bdcc62e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:32.737912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a82e84-8ce7-4c8b-bd45-a0a25bdcc62e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:32.738337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a82e84-8ce7-4c8b-bd45-a0a25bdcc62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:32.757224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f6a82e84-8ce7-4c8b-bd45-a0a25bdcc62e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:32.758606 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f6a82e84-8ce7-4c8b-bd45-a0a25bdcc62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:32.795585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ea329-9d03-4b23-9dd7-190395b1ed96 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:32.849128 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 484/966] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:32.853668 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c482e43-2477-42a2-a3e1-f9c1cdaa0c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:32.854728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c482e43-2477-42a2-a3e1-f9c1cdaa0c4a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:32.855167 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c482e43-2477-42a2-a3e1-f9c1cdaa0c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:32.855629 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c482e43-2477-42a2-a3e1-f9c1cdaa0c4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:32.864563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a82e84-8ce7-4c8b-bd45-a0a25bdcc62e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:32.867715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c482e43-2477-42a2-a3e1-f9c1cdaa0c4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:32.870244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c482e43-2477-42a2-a3e1-f9c1cdaa0c4a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:32.882757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:32.895005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-5c482e43-2477-42a2-a3e1-f9c1cdaa0c4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:32.895005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:32.895005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:32.895005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:32.895005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:32.895005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:32.895005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:32.895005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:32.895005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:32.895005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:32.895005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:32.895005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:32.895005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:32.895005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:32.895005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:32.897430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:32.898611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:32.898611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:32.898611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:32.898611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:32.901150 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 483/967] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:32.903553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c482e43-2477-42a2-a3e1-f9c1cdaa0c4a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:32.940678 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:32.988980 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a82e84-8ce7-4c8b-bd45-a0a25bdcc62e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:33.022706 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c482e43-2477-42a2-a3e1-f9c1cdaa0c4a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:33.026885 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f9e38-99bf-48ef-92ed-f6d5a1dd6c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:33.027926 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f9e38-99bf-48ef-92ed-f6d5a1dd6c88 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:33.028391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f9e38-99bf-48ef-92ed-f6d5a1dd6c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:33.028842 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f9e38-99bf-48ef-92ed-f6d5a1dd6c88 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:33.029704 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f9e38-99bf-48ef-92ed-f6d5a1dd6c88 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:33.031676 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f9e38-99bf-48ef-92ed-f6d5a1dd6c88 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-3c0f9e38-99bf-48ef-92ed-f6d5a1dd6c88 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:33.045117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:33.046235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:33.047109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:33.047109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:33.047109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:33.047109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:33.047109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:33.047109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:33.047109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:33.047109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:33.047109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:33.047109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:33.047109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:33.047109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:33.047109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:22:33.047109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:33.055125 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 485/968] 10.222.0.64 () {56 vars in 1126 bytes} [Mon May 23 15:22:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:33.056525 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8a1c8-fac4-4bae-931d-5155ce6a70ad None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:33.056812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8a1c8-fac4-4bae-931d-5155ce6a70ad None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:33.057110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8a1c8-fac4-4bae-931d-5155ce6a70ad None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:33.061245 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 486/969] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:33.061363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11dceb-0ff9-4b39-a176-18867631d450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:33.061486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8a1c8-fac4-4bae-931d-5155ce6a70ad None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:33.061564 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11dceb-0ff9-4b39-a176-18867631d450 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:33.061564 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11dceb-0ff9-4b39-a176-18867631d450 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:33.061564 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11dceb-0ff9-4b39-a176-18867631d450 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:33.061564 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11dceb-0ff9-4b39-a176-18867631d450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:33.061564 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11dceb-0ff9-4b39-a176-18867631d450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:33.062520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11dceb-0ff9-4b39-a176-18867631d450 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:33.075228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 484/970] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:33.076825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbbf31-2229-48f2-8f1f-7c1f47a29424 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:33.077149 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbbf31-2229-48f2-8f1f-7c1f47a29424 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:33.077439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbbf31-2229-48f2-8f1f-7c1f47a29424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:33.096447 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-05bbbf31-2229-48f2-8f1f-7c1f47a29424 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:33.097683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-05bbbf31-2229-48f2-8f1f-7c1f47a29424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:33.156614 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:33.168343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 485/971] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:33.177269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbbf31-2229-48f2-8f1f-7c1f47a29424 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:33.183888 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8a1c8-fac4-4bae-931d-5155ce6a70ad None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:33.217203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:33.226764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d487c-50ae-4a92-84e4-e071158915bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:33.227545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d487c-50ae-4a92-84e4-e071158915bf None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:33.229418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d487c-50ae-4a92-84e4-e071158915bf None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:33.229418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d487c-50ae-4a92-84e4-e071158915bf None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:33.229418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d487c-50ae-4a92-84e4-e071158915bf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:33.229418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d487c-50ae-4a92-84e4-e071158915bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:33.230953 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d487c-50ae-4a92-84e4-e071158915bf None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:33.240495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 487/972] 10.222.0.64 () {56 vars in 1122 bytes} [Mon May 23 15:22:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:33.240495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d487c-50ae-4a92-84e4-e071158915bf None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:33.272605 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbbf31-2229-48f2-8f1f-7c1f47a29424 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:33.310574 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec30a5-c6be-485a-93ef-96a3035b2a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:33.311413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec30a5-c6be-485a-93ef-96a3035b2a67 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:33.311677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec30a5-c6be-485a-93ef-96a3035b2a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:33.311955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec30a5-c6be-485a-93ef-96a3035b2a67 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:33.320007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec30a5-c6be-485a-93ef-96a3035b2a67 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:33.321871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec30a5-c6be-485a-93ef-96a3035b2a67 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-64ec30a5-c6be-485a-93ef-96a3035b2a67 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:22:33.337380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:33.338596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:33.339770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:33.339770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:33.339770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:22:33.339770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:22:33.339770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:22:33.339770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:22:33.339770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:22:33.339770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:22:33.339770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:22:33.339770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:33.339770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 486/973] 10.222.0.64 () {56 vars in 1117 bytes} [Mon May 23 15:22:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:33.339770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec30a5-c6be-485a-93ef-96a3035b2a67 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:33.356581 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d487c-50ae-4a92-84e4-e071158915bf None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:33.392497 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c561d-17f2-4264-9d21-ff59ead06ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:33.393222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c561d-17f2-4264-9d21-ff59ead06ba7 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:33.393511 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c561d-17f2-4264-9d21-ff59ead06ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:33.393798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c561d-17f2-4264-9d21-ff59ead06ba7 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:33.394818 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c561d-17f2-4264-9d21-ff59ead06ba7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:33.396246 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c561d-17f2-4264-9d21-ff59ead06ba7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:33.397570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: WARNING keystone.server.flask.common [None req-2b9c561d-17f2-4264-9d21-ff59ead06ba7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 23 15:22:33.401783 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:33.441716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.sql.core [None req-2b9c561d-17f2-4264-9d21-ff59ead06ba7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 15:22:33.441716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 15:22:33.441716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [parameters: {'user_id': 'fbfc36b5865f4dc0921242a87aab6b1c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 15:22:33.441716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80599) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 15:22:33.449299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-2b9c561d-17f2-4264-9d21-ff59ead06ba7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 15:22:33.449299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:33.449299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:22:33.449299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:22:33.449299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:22:33.449299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:22:33.449299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:22:33.449299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:22:33.449299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:22:33.449299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application conn.query(q) May 23 15:22:33.449299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:22:33.449299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:22:33.449299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:22:33.449299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application result.read() May 23 15:22:33.449299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:22:33.450358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application next(self.gen) May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application yield resource May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application next(self.gen) May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self.session.rollback() May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self.force_reraise() May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise self.value May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application session.commit() May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 15:22:33.451361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._prepare_impl() May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self.session.flush() May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._flush(objects) May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application compat.raise_( May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application flush_context.execute() May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rec.execute(self) May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 15:22:33.452274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application result = connection._execute_20( May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:22:33.453238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application conn.query(q) May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application result.read() May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:22:33.454183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:22:33.455055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:22:33.455055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:22:33.455055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:22:33.455055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:22:33.455055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 15:22:33.455055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 15:22:33.455055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fbfc36b5865f4dc0921242a87aab6b1c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 15:22:33.455055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 15:22:33.455055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:33.455055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:33.455055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:33.455055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:33.455055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:33.455055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:33.455055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 23 15:22:33.455870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 23 15:22:33.456959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 23 15:22:33.456959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 23 15:22:33.456959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 15:22:33.456959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 15:22:33.456959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 15:22:33.456959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:33.456959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 488/974] 10.222.0.64 () {60 vars in 1131 bytes} [Mon May 23 15:22:33 2022] POST /identity/v3/users => generated 160 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:33.456959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c561d-17f2-4264-9d21-ff59ead06ba7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:33.462302 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec30a5-c6be-485a-93ef-96a3035b2a67 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:33.501480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98dd59-4102-4ee5-bad2-e2b1829b3923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:33.502229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98dd59-4102-4ee5-bad2-e2b1829b3923 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:33.502520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98dd59-4102-4ee5-bad2-e2b1829b3923 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:33.502807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98dd59-4102-4ee5-bad2-e2b1829b3923 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:33.503512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98dd59-4102-4ee5-bad2-e2b1829b3923 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:33.503828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98dd59-4102-4ee5-bad2-e2b1829b3923 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:33.505248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98dd59-4102-4ee5-bad2-e2b1829b3923 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:33.514920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 487/975] 10.222.0.64 () {56 vars in 1122 bytes} [Mon May 23 15:22:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:33.514920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98dd59-4102-4ee5-bad2-e2b1829b3923 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:33.518758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:33.565506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c561d-17f2-4264-9d21-ff59ead06ba7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:33.600739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ffc7c-2ed7-4fa4-a211-01f5253b284b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:33.601531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ffc7c-2ed7-4fa4-a211-01f5253b284b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:33.601729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ffc7c-2ed7-4fa4-a211-01f5253b284b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:33.602020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ffc7c-2ed7-4fa4-a211-01f5253b284b None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:33.608247 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ffc7c-2ed7-4fa4-a211-01f5253b284b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:33.608247 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ffc7c-2ed7-4fa4-a211-01f5253b284b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:33.608247 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ffc7c-2ed7-4fa4-a211-01f5253b284b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:33.620037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98dd59-4102-4ee5-bad2-e2b1829b3923 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:33.632993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 489/976] 10.222.0.64 () {56 vars in 1149 bytes} [Mon May 23 15:22:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:33.632993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ffc7c-2ed7-4fa4-a211-01f5253b284b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:33.642910 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:33.663367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54262d13-cc6a-43cc-8356-43bfd956e139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:33.664163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54262d13-cc6a-43cc-8356-43bfd956e139 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:33.664451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54262d13-cc6a-43cc-8356-43bfd956e139 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:33.664727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54262d13-cc6a-43cc-8356-43bfd956e139 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:33.673917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54262d13-cc6a-43cc-8356-43bfd956e139 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:33.676043 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54262d13-cc6a-43cc-8356-43bfd956e139 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-54262d13-cc6a-43cc-8356-43bfd956e139 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:22:33.690744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:33.691917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:33.692919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:33.692919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:33.692919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:22:33.692919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:22:33.692919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:22:33.692919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:22:33.692919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:22:33.692919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:22:33.692919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:22:33.692919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:33.692919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 488/977] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:22:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:33.692919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54262d13-cc6a-43cc-8356-43bfd956e139 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:33.736160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ffc7c-2ed7-4fa4-a211-01f5253b284b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:33.748911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:33.769899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88cee6-3430-4bef-b930-eb983531990a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:33.770553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88cee6-3430-4bef-b930-eb983531990a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:33.770846 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88cee6-3430-4bef-b930-eb983531990a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:33.771129 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88cee6-3430-4bef-b930-eb983531990a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:33.772020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88cee6-3430-4bef-b930-eb983531990a None admin] RBAC: Adding query filter params (interface=admin, service_id=c25a7fa74aeb428bb823ff97514e3bb2, region_id=RegionOne) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:33.772594 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88cee6-3430-4bef-b930-eb983531990a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:33.774131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88cee6-3430-4bef-b930-eb983531990a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:33.779555 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 490/978] 10.222.0.64 () {56 vars in 1249 bytes} [Mon May 23 15:22:33 2022] GET /identity/v3/endpoints?service_id=c25a7fa74aeb428bb823ff97514e3bb2&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:33.780910 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20785374-87b6-4c1e-8059-cb395608ac4a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:33.781233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20785374-87b6-4c1e-8059-cb395608ac4a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:33.781516 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20785374-87b6-4c1e-8059-cb395608ac4a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:33.782779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 491/979] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:33.782779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20785374-87b6-4c1e-8059-cb395608ac4a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:33.798784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54262d13-cc6a-43cc-8356-43bfd956e139 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:33.832402 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef82c1-c97a-4334-a59f-fb1900537e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:33.833050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef82c1-c97a-4334-a59f-fb1900537e9f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:33.833359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef82c1-c97a-4334-a59f-fb1900537e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:33.833738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef82c1-c97a-4334-a59f-fb1900537e9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:33.874887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:33.886420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20785374-87b6-4c1e-8059-cb395608ac4a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:33.900178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef82c1-c97a-4334-a59f-fb1900537e9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:33.901761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef82c1-c97a-4334-a59f-fb1900537e9f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:33.920204 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01943b2d-2d0f-424c-9c47-331fc58548e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:33.920867 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01943b2d-2d0f-424c-9c47-331fc58548e1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:33.921182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01943b2d-2d0f-424c-9c47-331fc58548e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:33.921455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01943b2d-2d0f-424c-9c47-331fc58548e1 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:33.922247 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01943b2d-2d0f-424c-9c47-331fc58548e1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:33.923098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01943b2d-2d0f-424c-9c47-331fc58548e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:33.924645 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01943b2d-2d0f-424c-9c47-331fc58548e1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:33.958432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 492/980] 10.222.0.64 () {56 vars in 1101 bytes} [Mon May 23 15:22:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:33.959243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01943b2d-2d0f-424c-9c47-331fc58548e1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:34.006051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 489/981] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:22:33 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:22:34.007693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90b92d-ae3f-49a9-a320-9dc5471365d7 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:34.007988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90b92d-ae3f-49a9-a320-9dc5471365d7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:34.008269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90b92d-ae3f-49a9-a320-9dc5471365d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:34.018403 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:34.027814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8e90b92d-ae3f-49a9-a320-9dc5471365d7 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:34.028954 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8e90b92d-ae3f-49a9-a320-9dc5471365d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:34.096797 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01943b2d-2d0f-424c-9c47-331fc58548e1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:34.105405 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 490/982] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:34.106273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90b92d-ae3f-49a9-a320-9dc5471365d7 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:34.109549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:34.147794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2939e-2e67-4a5e-bf07-79954ecdfb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:34.148467 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2939e-2e67-4a5e-bf07-79954ecdfb4e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:34.148751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2939e-2e67-4a5e-bf07-79954ecdfb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:34.149028 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2939e-2e67-4a5e-bf07-79954ecdfb4e None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:34.150103 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2939e-2e67-4a5e-bf07-79954ecdfb4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:34.151090 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2939e-2e67-4a5e-bf07-79954ecdfb4e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:34.158299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 493/983] 10.222.0.64 () {56 vars in 1087 bytes} [Mon May 23 15:22:33 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:22:34.160526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2939e-2e67-4a5e-bf07-79954ecdfb4e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:34.243119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90b92d-ae3f-49a9-a320-9dc5471365d7 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:34.285461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad8433-8265-4a32-bc15-0609d9390471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:34.286290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad8433-8265-4a32-bc15-0609d9390471 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:34.286575 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad8433-8265-4a32-bc15-0609d9390471 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:34.286851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad8433-8265-4a32-bc15-0609d9390471 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:34.295477 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:34.300704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad8433-8265-4a32-bc15-0609d9390471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:34.301044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2939e-2e67-4a5e-bf07-79954ecdfb4e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:34.302510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad8433-8265-4a32-bc15-0609d9390471 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:34.322676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-3fad8433-8265-4a32-bc15-0609d9390471 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:34.322676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:34.322676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:34.322676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:34.322676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:34.322676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:34.322676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:34.322676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:34.322676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:34.322676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:34.322676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:34.322676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:34.322676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:34.322676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:34.322676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:34.323676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:34.324702 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:34.324702 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:34.324702 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:34.324702 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:34.328948 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 491/984] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:34.329773 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad8433-8265-4a32-bc15-0609d9390471 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:34.341402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e388241-97ad-4796-9cc4-82bb6b4b7253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:34.342561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e388241-97ad-4796-9cc4-82bb6b4b7253 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:34.343201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e388241-97ad-4796-9cc4-82bb6b4b7253 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:34.343673 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e388241-97ad-4796-9cc4-82bb6b4b7253 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:34.344569 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e388241-97ad-4796-9cc4-82bb6b4b7253 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:34.346217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e388241-97ad-4796-9cc4-82bb6b4b7253 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:34.390682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:34.420001 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 494/985] 10.222.0.64 () {60 vars in 1141 bytes} [Mon May 23 15:22:34 2022] POST /identity/v3/services => generated 242 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:22:34.435943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad8433-8265-4a32-bc15-0609d9390471 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:34.482764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f59e02-e609-48ac-aca3-30385de3ed82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:34.483465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f59e02-e609-48ac-aca3-30385de3ed82 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:34.483748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f59e02-e609-48ac-aca3-30385de3ed82 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:34.484025 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f59e02-e609-48ac-aca3-30385de3ed82 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:34.484762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f59e02-e609-48ac-aca3-30385de3ed82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:34.485763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f59e02-e609-48ac-aca3-30385de3ed82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:34.487776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f59e02-e609-48ac-aca3-30385de3ed82 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:34.505976 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 492/986] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:34.507410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e388241-97ad-4796-9cc4-82bb6b4b7253 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:34.619644 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e388241-97ad-4796-9cc4-82bb6b4b7253 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:34.657535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08170b3-6b21-4160-8121-90bbb72225eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:34.658372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08170b3-6b21-4160-8121-90bbb72225eb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:34.658661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08170b3-6b21-4160-8121-90bbb72225eb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:34.658941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08170b3-6b21-4160-8121-90bbb72225eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:34.666006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08170b3-6b21-4160-8121-90bbb72225eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:34.667449 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08170b3-6b21-4160-8121-90bbb72225eb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:34.682616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a08170b3-6b21-4160-8121-90bbb72225eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:34.682616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:34.682616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:34.682616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:34.682616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:34.682616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:34.682616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:34.682616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:34.682616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:34.682616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:34.682616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:34.682616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:34.682616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:34.682616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:34.682616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:34.683556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:34.684489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:34.684489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:34.684489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:34.684489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:34.684489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:34.684489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:34.684489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:34.684489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:34.684489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 495/987] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:34.691199 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f59e02-e609-48ac-aca3-30385de3ed82 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:34.800484 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f59e02-e609-48ac-aca3-30385de3ed82 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:34.814518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:34.836262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68309126-5a69-4a98-891d-48effcb0a57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:34.836934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68309126-5a69-4a98-891d-48effcb0a57a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:34.837258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68309126-5a69-4a98-891d-48effcb0a57a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:34.837542 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68309126-5a69-4a98-891d-48effcb0a57a None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:34.838340 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68309126-5a69-4a98-891d-48effcb0a57a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:34.838657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68309126-5a69-4a98-891d-48effcb0a57a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:34.840108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68309126-5a69-4a98-891d-48effcb0a57a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:34.853001 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 493/988] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:34.858786 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08170b3-6b21-4160-8121-90bbb72225eb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:34.878430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:34.961051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08170b3-6b21-4160-8121-90bbb72225eb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:34.995330 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b01a4a-dc38-4bbe-881b-9d583dd29d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:34.995998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b01a4a-dc38-4bbe-881b-9d583dd29d25 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:34.996289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b01a4a-dc38-4bbe-881b-9d583dd29d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:34.996617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b01a4a-dc38-4bbe-881b-9d583dd29d25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:34.997484 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b01a4a-dc38-4bbe-881b-9d583dd29d25 None admin] RBAC: Adding query filter params (role.id=91ddccb742f94d19958d422cab370fa3, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=08dad0888c8542f4aed748a1a35db2c4) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:34.997802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b01a4a-dc38-4bbe-881b-9d583dd29d25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:34.999227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b01a4a-dc38-4bbe-881b-9d583dd29d25 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:35.009639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 496/989] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:22:34 2022] GET /identity/v3/role_assignments?user.id=08dad0888c8542f4aed748a1a35db2c4&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=91ddccb742f94d19958d422cab370fa3 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:36.401401 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e67534-e131-48ad-bd32-ea5930d460f6 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:36.401934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e67534-e131-48ad-bd32-ea5930d460f6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:36.402016 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e67534-e131-48ad-bd32-ea5930d460f6 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:36.403488 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 494/990] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:36.409020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8be0a-be39-44ca-ba63-6336ebf43c19 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:36.409505 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8be0a-be39-44ca-ba63-6336ebf43c19 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:36.409703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8be0a-be39-44ca-ba63-6336ebf43c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:36.426562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-74e8be0a-be39-44ca-ba63-6336ebf43c19 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:36.428050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-74e8be0a-be39-44ca-ba63-6336ebf43c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:36.499888 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 497/991] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:36.523269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e67534-e131-48ad-bd32-ea5930d460f6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:36.625897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e67534-e131-48ad-bd32-ea5930d460f6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:36.661035 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8737a03-14d3-4353-8b96-4d094c077692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:36.661800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8737a03-14d3-4353-8b96-4d094c077692 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:36.662085 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8737a03-14d3-4353-8b96-4d094c077692 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:36.662363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8737a03-14d3-4353-8b96-4d094c077692 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:36.663017 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8737a03-14d3-4353-8b96-4d094c077692 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:36.664516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8737a03-14d3-4353-8b96-4d094c077692 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:36.677453 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8e9b0-9704-4c96-8f0f-f17ec0a6ac6b None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-f8737a03-14d3-4353-8b96-4d094c077692 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:36.677621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:36.678583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:36.679446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:36.679446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:36.679446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:36.679446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:36.679446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:36.679446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:36.679446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:36.679446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:36.679446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:36.679446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:36.679446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:36.679446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:36.679446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:22:36.679446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:36.680209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8e9b0-9704-4c96-8f0f-f17ec0a6ac6b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:36.680209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8e9b0-9704-4c96-8f0f-f17ec0a6ac6b None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:36.680209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 498/992] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:36.681347 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 495/993] 10.222.0.64 () {56 vars in 1126 bytes} [Mon May 23 15:22:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:36.684721 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8737a03-14d3-4353-8b96-4d094c077692 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:36.684855 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5ca44-fa95-421e-8e60-9aa99c607bd9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:36.685033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5ca44-fa95-421e-8e60-9aa99c607bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:36.685345 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5ca44-fa95-421e-8e60-9aa99c607bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:36.702660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-96b5ca44-fa95-421e-8e60-9aa99c607bd9 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:36.703927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-96b5ca44-fa95-421e-8e60-9aa99c607bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:36.711714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:36.730764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:36.772869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 499/994] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:36.795161 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5ca44-fa95-421e-8e60-9aa99c607bd9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:36.801808 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8737a03-14d3-4353-8b96-4d094c077692 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:36.839135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5fba0-43c7-49af-a408-01580cce46e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:36.839805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5fba0-43c7-49af-a408-01580cce46e2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:36.840091 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5fba0-43c7-49af-a408-01580cce46e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:36.840369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5fba0-43c7-49af-a408-01580cce46e2 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:36.841093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5fba0-43c7-49af-a408-01580cce46e2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:36.841415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5fba0-43c7-49af-a408-01580cce46e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:36.842804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5fba0-43c7-49af-a408-01580cce46e2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:36.848903 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 496/995] 10.222.0.64 () {56 vars in 1122 bytes} [Mon May 23 15:22:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:36.854535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5fba0-43c7-49af-a408-01580cce46e2 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:36.903941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5ca44-fa95-421e-8e60-9aa99c607bd9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:36.943094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94f80e-4111-406a-aa1e-2a5163b96e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:36.944247 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94f80e-4111-406a-aa1e-2a5163b96e5e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:36.944835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94f80e-4111-406a-aa1e-2a5163b96e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:36.945296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94f80e-4111-406a-aa1e-2a5163b96e5e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:36.946348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94f80e-4111-406a-aa1e-2a5163b96e5e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:36.947855 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94f80e-4111-406a-aa1e-2a5163b96e5e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:36.957451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:36.960799 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5fba0-43c7-49af-a408-01580cce46e2 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-2e94f80e-4111-406a-aa1e-2a5163b96e5e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:36.964186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:36.965460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:36.973396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:36.973396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:36.973396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:36.973396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:36.973396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:36.973396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:36.973396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:36.973396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:36.973396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:36.973396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:36.973396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:36.973396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:36.973396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:22:36.973396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:36.973396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 500/996] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:36.974398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94f80e-4111-406a-aa1e-2a5163b96e5e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:36.983506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:37.001262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71613d31-78d6-4d38-b3c1-b269a75ad75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:37.002423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71613d31-78d6-4d38-b3c1-b269a75ad75c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:37.003029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71613d31-78d6-4d38-b3c1-b269a75ad75c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:37.003907 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71613d31-78d6-4d38-b3c1-b269a75ad75c None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:37.005104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71613d31-78d6-4d38-b3c1-b269a75ad75c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:37.005762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71613d31-78d6-4d38-b3c1-b269a75ad75c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:37.007899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71613d31-78d6-4d38-b3c1-b269a75ad75c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:37.014422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 497/997] 10.222.0.64 () {56 vars in 1122 bytes} [Mon May 23 15:22:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:37.020389 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71613d31-78d6-4d38-b3c1-b269a75ad75c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:37.087821 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94f80e-4111-406a-aa1e-2a5163b96e5e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:37.126020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc1868-5250-462f-8e4f-6b04472c45b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:37.126829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc1868-5250-462f-8e4f-6b04472c45b2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:37.127115 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc1868-5250-462f-8e4f-6b04472c45b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:37.127387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc1868-5250-462f-8e4f-6b04472c45b2 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:37.128091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc1868-5250-462f-8e4f-6b04472c45b2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:37.128410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc1868-5250-462f-8e4f-6b04472c45b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:37.129873 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc1868-5250-462f-8e4f-6b04472c45b2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:37.130818 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71613d31-78d6-4d38-b3c1-b269a75ad75c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:37.135119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 501/998] 10.222.0.64 () {56 vars in 1112 bytes} [Mon May 23 15:22:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:37.141482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc1868-5250-462f-8e4f-6b04472c45b2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:37.170545 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fee2f-157f-4dec-8e05-5f1845100293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:37.171210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fee2f-157f-4dec-8e05-5f1845100293 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:37.171493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fee2f-157f-4dec-8e05-5f1845100293 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:37.171770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fee2f-157f-4dec-8e05-5f1845100293 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:37.173174 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1fee2f-157f-4dec-8e05-5f1845100293 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:37.174638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1fee2f-157f-4dec-8e05-5f1845100293 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:37.215109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:37.242448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 498/999] 10.222.0.64 () {60 vars in 1144 bytes} [Mon May 23 15:22:37 2022] POST /identity/v3/endpoints => generated 325 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:22:37.243633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbe482-de43-47aa-af66-128547e84af8 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:37.243932 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbe482-de43-47aa-af66-128547e84af8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:37.244208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbe482-de43-47aa-af66-128547e84af8 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:37.245883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc1868-5250-462f-8e4f-6b04472c45b2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:37.249672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 499/1000] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:37.252056 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885181f-cdbe-4747-b855-74cb782cd833 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:37.252354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885181f-cdbe-4747-b855-74cb782cd833 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:37.252802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885181f-cdbe-4747-b855-74cb782cd833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:37.271246 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f885181f-cdbe-4747-b855-74cb782cd833 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:37.272493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f885181f-cdbe-4747-b855-74cb782cd833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:37.291338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:37.298899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328db462-829a-4a44-bf98-8607f3970242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:37.299877 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328db462-829a-4a44-bf98-8607f3970242 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:37.300395 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328db462-829a-4a44-bf98-8607f3970242 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:37.300827 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328db462-829a-4a44-bf98-8607f3970242 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:37.302160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328db462-829a-4a44-bf98-8607f3970242 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:37.302474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328db462-829a-4a44-bf98-8607f3970242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:37.303890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328db462-829a-4a44-bf98-8607f3970242 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:37.315704 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 502/1001] 10.222.0.64 () {56 vars in 1112 bytes} [Mon May 23 15:22:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:37.316997 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328db462-829a-4a44-bf98-8607f3970242 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:37.353232 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 500/1002] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:37.374125 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885181f-cdbe-4747-b855-74cb782cd833 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:37.442011 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328db462-829a-4a44-bf98-8607f3970242 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:37.478849 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:37.486045 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f838f4-88e7-4937-9573-90fee3dc9955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:37.486716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f838f4-88e7-4937-9573-90fee3dc9955 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:37.487005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f838f4-88e7-4937-9573-90fee3dc9955 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:37.487285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f838f4-88e7-4937-9573-90fee3dc9955 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:37.492854 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f838f4-88e7-4937-9573-90fee3dc9955 None admin] RBAC: Adding query filter params (interface=public, service_id=29f319324c3246308de77006d058037d, region_id=RegionOne) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:37.492854 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f838f4-88e7-4937-9573-90fee3dc9955 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:37.492854 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f838f4-88e7-4937-9573-90fee3dc9955 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:37.495463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 503/1003] 10.222.0.64 () {56 vars in 1251 bytes} [Mon May 23 15:22:37 2022] GET /identity/v3/endpoints?service_id=29f319324c3246308de77006d058037d&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:37.501274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f838f4-88e7-4937-9573-90fee3dc9955 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:37.506880 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885181f-cdbe-4747-b855-74cb782cd833 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:37.543742 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a1ed0-2add-4424-830c-914392ecb059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:37.544538 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a1ed0-2add-4424-830c-914392ecb059 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:37.544820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a1ed0-2add-4424-830c-914392ecb059 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:37.545121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a1ed0-2add-4424-830c-914392ecb059 None admin] PATH_INFO: `/v3/services/image` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:37.545775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a1ed0-2add-4424-830c-914392ecb059 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:37.547139 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a1ed0-2add-4424-830c-914392ecb059 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-945a1ed0-2add-4424-830c-914392ecb059 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:37.559136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:37.560196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:37.561110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:37.561110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:37.561110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:37.561110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:37.561110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:37.561110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:37.561110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:37.561110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:37.561110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:37.561110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:37.561110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:37.561110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:37.561110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:22:37.561110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:37.565548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 501/1004] 10.222.0.64 () {56 vars in 1105 bytes} [Mon May 23 15:22:37 2022] GET /identity/v3/services/image => generated 86 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:37.566303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a1ed0-2add-4424-830c-914392ecb059 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:37.586254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:37.614926 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f838f4-88e7-4937-9573-90fee3dc9955 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:37.651303 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0e2ed-fc22-4f49-8c7e-57da628d7aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:37.652083 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0e2ed-fc22-4f49-8c7e-57da628d7aba None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:37.652308 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0e2ed-fc22-4f49-8c7e-57da628d7aba None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:37.652590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0e2ed-fc22-4f49-8c7e-57da628d7aba None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:37.653453 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0e2ed-fc22-4f49-8c7e-57da628d7aba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:37.654848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0e2ed-fc22-4f49-8c7e-57da628d7aba None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:37.662201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 504/1005] 10.222.0.64 () {56 vars in 1087 bytes} [Mon May 23 15:22:37 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:22:37.676837 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a1ed0-2add-4424-830c-914392ecb059 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:37.721734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565f3c1-cd72-4fe7-8ff9-9767d551dd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:37.722432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565f3c1-cd72-4fe7-8ff9-9767d551dd96 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:37.722715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565f3c1-cd72-4fe7-8ff9-9767d551dd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:37.722991 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565f3c1-cd72-4fe7-8ff9-9767d551dd96 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:37.723756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565f3c1-cd72-4fe7-8ff9-9767d551dd96 None admin] RBAC: Adding query filter params (name=image) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:37.724068 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565f3c1-cd72-4fe7-8ff9-9767d551dd96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:37.725533 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565f3c1-cd72-4fe7-8ff9-9767d551dd96 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:37.731695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 502/1006] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:37.738463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0e2ed-fc22-4f49-8c7e-57da628d7aba None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:37.807020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:37.852387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0e2ed-fc22-4f49-8c7e-57da628d7aba None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:37.888833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2853f-ff43-4204-9d4b-e96e6f936d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:37.889539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2853f-ff43-4204-9d4b-e96e6f936d0f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:37.889824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2853f-ff43-4204-9d4b-e96e6f936d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:37.890102 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2853f-ff43-4204-9d4b-e96e6f936d0f None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:37.890809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2853f-ff43-4204-9d4b-e96e6f936d0f None admin] RBAC: Adding query filter params (type=image) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:37.891685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2853f-ff43-4204-9d4b-e96e6f936d0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:37.893038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2853f-ff43-4204-9d4b-e96e6f936d0f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:37.902914 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 505/1007] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:37 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:39.559210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8198f8-3b0b-45f8-b4e8-ade800dc54ae None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:39.559765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8198f8-3b0b-45f8-b4e8-ade800dc54ae None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:39.559765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8198f8-3b0b-45f8-b4e8-ade800dc54ae None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:39.561319 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 503/1008] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:39.566948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a59b9-a151-4ec5-9598-a703a977d2d3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:39.567359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a59b9-a151-4ec5-9598-a703a977d2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:39.567524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a59b9-a151-4ec5-9598-a703a977d2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:39.584982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ad7a59b9-a151-4ec5-9598-a703a977d2d3 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:39.586194 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ad7a59b9-a151-4ec5-9598-a703a977d2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:39.653826 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 506/1009] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:39.677639 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8198f8-3b0b-45f8-b4e8-ade800dc54ae None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:39.780365 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:39.806262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8198f8-3b0b-45f8-b4e8-ade800dc54ae None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:39.841211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcb3c4-a15a-46fa-bf8a-2fad6730dad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:39.841931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcb3c4-a15a-46fa-bf8a-2fad6730dad8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:39.842219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcb3c4-a15a-46fa-bf8a-2fad6730dad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:39.842500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcb3c4-a15a-46fa-bf8a-2fad6730dad8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:39.843698 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcb3c4-a15a-46fa-bf8a-2fad6730dad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:39.845232 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcb3c4-a15a-46fa-bf8a-2fad6730dad8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:39.853273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 504/1010] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:39.853433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a59b9-a151-4ec5-9598-a703a977d2d3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:39.893520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a58298-fd4e-48b2-b530-74da77f18241 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:39.893872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a58298-fd4e-48b2-b530-74da77f18241 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:39.894089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a58298-fd4e-48b2-b530-74da77f18241 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:39.895587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 505/1011] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:39.900261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81dbfa-f8e6-4e69-a789-94a1717aa3e4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:39.900551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81dbfa-f8e6-4e69-a789-94a1717aa3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:39.900832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81dbfa-f8e6-4e69-a789-94a1717aa3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:39.912150 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:39.918258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-1d81dbfa-f8e6-4e69-a789-94a1717aa3e4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:39.919450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1d81dbfa-f8e6-4e69-a789-94a1717aa3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:39.967978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a59b9-a151-4ec5-9598-a703a977d2d3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:40.002147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 506/1012] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:40.007592 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e024b8e-2e9f-476e-b1eb-942ffbb3ff4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:40.008263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e024b8e-2e9f-476e-b1eb-942ffbb3ff4c None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:40.008549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e024b8e-2e9f-476e-b1eb-942ffbb3ff4c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:40.008830 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e024b8e-2e9f-476e-b1eb-942ffbb3ff4c None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:40.009636 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e024b8e-2e9f-476e-b1eb-942ffbb3ff4c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:40.011156 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e024b8e-2e9f-476e-b1eb-942ffbb3ff4c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:40.024440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81dbfa-f8e6-4e69-a789-94a1717aa3e4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:40.088613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 507/1013] 10.222.0.64 () {60 vars in 1141 bytes} [Mon May 23 15:22:39 2022] POST /identity/v3/projects => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:22:40.115692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:40.136893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81dbfa-f8e6-4e69-a789-94a1717aa3e4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:40.137689 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6be98-83a5-4c58-a37a-206bbfd3a656 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:40.137987 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6be98-83a5-4c58-a37a-206bbfd3a656 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:40.138274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6be98-83a5-4c58-a37a-206bbfd3a656 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:40.139663 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 508/1014] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:40.145868 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097e540-5489-4ca3-8831-833499e0fcfa None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:40.146145 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097e540-5489-4ca3-8831-833499e0fcfa None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:40.146432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097e540-5489-4ca3-8831-833499e0fcfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:40.165216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d097e540-5489-4ca3-8831-833499e0fcfa None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:40.166446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d097e540-5489-4ca3-8831-833499e0fcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:40.183254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dea13c-4027-4986-89d1-5a3788455e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:40.184068 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dea13c-4027-4986-89d1-5a3788455e98 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:40.184355 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dea13c-4027-4986-89d1-5a3788455e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:40.184638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dea13c-4027-4986-89d1-5a3788455e98 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:40.185331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dea13c-4027-4986-89d1-5a3788455e98 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:40.186740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dea13c-4027-4986-89d1-5a3788455e98 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-70dea13c-4027-4986-89d1-5a3788455e98 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:40.199823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:40.200943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:40.202499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:40.202499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:40.202499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:40.202499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:40.202499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:40.202499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:40.202499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:40.202499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:40.202499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:40.202499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:40.202499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:40.202499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:40.202499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:22:40.202499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:40.202499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 507/1015] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:40.207354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dea13c-4027-4986-89d1-5a3788455e98 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:40.247617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 509/1016] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:40 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:40.270907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097e540-5489-4ca3-8831-833499e0fcfa None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:40.307830 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:40.323468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dea13c-4027-4986-89d1-5a3788455e98 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:40.363018 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf4e97-f2ab-47ba-965d-49a838c41509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:40.363705 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf4e97-f2ab-47ba-965d-49a838c41509 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:40.363995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf4e97-f2ab-47ba-965d-49a838c41509 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:40.364275 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf4e97-f2ab-47ba-965d-49a838c41509 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:40.364989 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf4e97-f2ab-47ba-965d-49a838c41509 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:40.365335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf4e97-f2ab-47ba-965d-49a838c41509 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:40.366810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf4e97-f2ab-47ba-965d-49a838c41509 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:40.372343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 508/1017] 10.222.0.64 () {56 vars in 1112 bytes} [Mon May 23 15:22:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:40.378540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf4e97-f2ab-47ba-965d-49a838c41509 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:40.387937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097e540-5489-4ca3-8831-833499e0fcfa None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:40.389765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:40.424034 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c2f68-4a8e-4a70-ba63-daea9f06ee52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:40.424712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c2f68-4a8e-4a70-ba63-daea9f06ee52 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:40.424996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c2f68-4a8e-4a70-ba63-daea9f06ee52 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:40.425307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c2f68-4a8e-4a70-ba63-daea9f06ee52 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:40.426295 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c2f68-4a8e-4a70-ba63-daea9f06ee52 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:40.427664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c2f68-4a8e-4a70-ba63-daea9f06ee52 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:40.482011 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf4e97-f2ab-47ba-965d-49a838c41509 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:40.517448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 510/1018] 10.222.0.64 () {60 vars in 1141 bytes} [Mon May 23 15:22:40 2022] POST /identity/v3/services => generated 242 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:22:40.524310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f0b09-a8dc-40d0-a4ea-bd26a70f77e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:40.525087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f0b09-a8dc-40d0-a4ea-bd26a70f77e2 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:40.525393 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f0b09-a8dc-40d0-a4ea-bd26a70f77e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:40.525974 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f0b09-a8dc-40d0-a4ea-bd26a70f77e2 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:40.526761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f0b09-a8dc-40d0-a4ea-bd26a70f77e2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:40.527140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f0b09-a8dc-40d0-a4ea-bd26a70f77e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:40.528726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f0b09-a8dc-40d0-a4ea-bd26a70f77e2 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:40.534958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 509/1019] 10.222.0.64 () {56 vars in 1112 bytes} [Mon May 23 15:22:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:40.540532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c2f68-4a8e-4a70-ba63-daea9f06ee52 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:40.649945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:40.659756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c2f68-4a8e-4a70-ba63-daea9f06ee52 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:40.709485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f56e5-aa3c-404c-bd6a-4557b3fe39c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:40.710174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f56e5-aa3c-404c-bd6a-4557b3fe39c7 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:40.710461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f56e5-aa3c-404c-bd6a-4557b3fe39c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:40.710739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f56e5-aa3c-404c-bd6a-4557b3fe39c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:40.711520 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f56e5-aa3c-404c-bd6a-4557b3fe39c7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:40.712913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f56e5-aa3c-404c-bd6a-4557b3fe39c7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:40.796702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 511/1020] 10.222.0.64 () {60 vars in 1144 bytes} [Mon May 23 15:22:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:22:42.395944 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e4a8c-bc69-41ed-9eaa-483046fb8c26 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:42.396485 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e4a8c-bc69-41ed-9eaa-483046fb8c26 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:42.396574 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e4a8c-bc69-41ed-9eaa-483046fb8c26 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:42.398076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 510/1021] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:42.403419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd87f12-2c8a-4067-8e82-6f16ee6428a9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:42.403825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd87f12-2c8a-4067-8e82-6f16ee6428a9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:42.404004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd87f12-2c8a-4067-8e82-6f16ee6428a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:42.420902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-acd87f12-2c8a-4067-8e82-6f16ee6428a9 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:42.422139 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-acd87f12-2c8a-4067-8e82-6f16ee6428a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:42.497429 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 512/1022] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:42.522395 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e4a8c-bc69-41ed-9eaa-483046fb8c26 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:42.594721 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:42.630418 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e4a8c-bc69-41ed-9eaa-483046fb8c26 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:42.665484 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600bc4f-734c-455c-b02a-7294c5bf71dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:42.666197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600bc4f-734c-455c-b02a-7294c5bf71dd None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:42.666481 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600bc4f-734c-455c-b02a-7294c5bf71dd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:42.666753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600bc4f-734c-455c-b02a-7294c5bf71dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:42.667919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600bc4f-734c-455c-b02a-7294c5bf71dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:42.669426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600bc4f-734c-455c-b02a-7294c5bf71dd None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:42.675739 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 511/1023] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:42.677670 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd87f12-2c8a-4067-8e82-6f16ee6428a9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:42.757764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225a0c8-6c53-48cb-a710-0614a36cdcf3 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:42.757764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225a0c8-6c53-48cb-a710-0614a36cdcf3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:42.757764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225a0c8-6c53-48cb-a710-0614a36cdcf3 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:42.759117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 512/1024] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:42.765342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47d1f0-a997-4869-b778-acf6ec8423a1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:42.766306 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47d1f0-a997-4869-b778-acf6ec8423a1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:42.766711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47d1f0-a997-4869-b778-acf6ec8423a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:42.774840 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:42.786807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8f47d1f0-a997-4869-b778-acf6ec8423a1 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:42.788050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8f47d1f0-a997-4869-b778-acf6ec8423a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:42.802994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd87f12-2c8a-4067-8e82-6f16ee6428a9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:42.843296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315508e-6553-4a6b-b7f0-a52296d5e3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:42.849433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315508e-6553-4a6b-b7f0-a52296d5e3ed None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:42.849433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315508e-6553-4a6b-b7f0-a52296d5e3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:42.849433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315508e-6553-4a6b-b7f0-a52296d5e3ed None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:42.849433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315508e-6553-4a6b-b7f0-a52296d5e3ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:42.849433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315508e-6553-4a6b-b7f0-a52296d5e3ed None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:42.860337 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 513/1025] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:42.883834 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47d1f0-a997-4869-b778-acf6ec8423a1 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:42.949004 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:42.986890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47d1f0-a997-4869-b778-acf6ec8423a1 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:43.020561 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66bbfd-d008-4726-aeb0-93bf4ce6a05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:43.021387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66bbfd-d008-4726-aeb0-93bf4ce6a05e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:43.021733 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66bbfd-d008-4726-aeb0-93bf4ce6a05e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:43.022017 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66bbfd-d008-4726-aeb0-93bf4ce6a05e None admin] PATH_INFO: `/v3/services/image` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:43.022674 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66bbfd-d008-4726-aeb0-93bf4ce6a05e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:43.024053 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66bbfd-d008-4726-aeb0-93bf4ce6a05e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-1a66bbfd-d008-4726-aeb0-93bf4ce6a05e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:43.036394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:43.037540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:43.038524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:43.038524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:43.038524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:43.038524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:43.038524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:43.038524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:43.038524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:43.038524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:43.038524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:43.038524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:43.038524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:43.038524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:43.038524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:22:43.038524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:43.038524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 514/1026] 10.222.0.64 () {56 vars in 1105 bytes} [Mon May 23 15:22:42 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:43.043452 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66bbfd-d008-4726-aeb0-93bf4ce6a05e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:43.043865 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 513/1027] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:22:42 2022] POST /identity/v3/users => generated 285 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:22:43.152248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3098b08-5c66-4c80-b066-1ff68743b210 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:43.152538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3098b08-5c66-4c80-b066-1ff68743b210 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:43.152822 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3098b08-5c66-4c80-b066-1ff68743b210 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:43.154432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 514/1028] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:43.161092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d135b-47af-488e-b96a-0c01532341f1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:43.161430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d135b-47af-488e-b96a-0c01532341f1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:43.161687 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d135b-47af-488e-b96a-0c01532341f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:43.162287 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66bbfd-d008-4726-aeb0-93bf4ce6a05e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:43.181839 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-547d135b-47af-488e-b96a-0c01532341f1 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:43.183079 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-547d135b-47af-488e-b96a-0c01532341f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:43.194046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:43.200087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001f7e9-4d4d-4af8-8f68-60c439d61533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:43.200699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001f7e9-4d4d-4af8-8f68-60c439d61533 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:43.200992 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001f7e9-4d4d-4af8-8f68-60c439d61533 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:43.201311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001f7e9-4d4d-4af8-8f68-60c439d61533 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:43.202037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001f7e9-4d4d-4af8-8f68-60c439d61533 None admin] RBAC: Adding query filter params (name=image) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:43.202423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001f7e9-4d4d-4af8-8f68-60c439d61533 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:43.204009 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001f7e9-4d4d-4af8-8f68-60c439d61533 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:43.209754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 515/1029] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:43.216292 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001f7e9-4d4d-4af8-8f68-60c439d61533 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:43.279374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 515/1030] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:43.306493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d135b-47af-488e-b96a-0c01532341f1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:43.329962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001f7e9-4d4d-4af8-8f68-60c439d61533 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:43.366737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf3425-97c8-404a-b6f9-028d64a709cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:43.367410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf3425-97c8-404a-b6f9-028d64a709cf None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:43.367695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf3425-97c8-404a-b6f9-028d64a709cf None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:43.367975 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf3425-97c8-404a-b6f9-028d64a709cf None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:43.368677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf3425-97c8-404a-b6f9-028d64a709cf None admin] RBAC: Adding query filter params (type=image) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:43.368987 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf3425-97c8-404a-b6f9-028d64a709cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:43.370458 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf3425-97c8-404a-b6f9-028d64a709cf None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:43.375618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 516/1031] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:43.376555 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:43.381797 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf3425-97c8-404a-b6f9-028d64a709cf None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:43.427022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d135b-47af-488e-b96a-0c01532341f1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:43.443571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:43.464800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfeb15-03b2-4bd2-b6cc-54158ef7302d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:43.465653 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfeb15-03b2-4bd2-b6cc-54158ef7302d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:43.465948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfeb15-03b2-4bd2-b6cc-54158ef7302d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:43.466229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfeb15-03b2-4bd2-b6cc-54158ef7302d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:43.478199 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfeb15-03b2-4bd2-b6cc-54158ef7302d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-39cfeb15-03b2-4bd2-b6cc-54158ef7302d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:43.486551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:43.487560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:43.490080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:43.490080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:43.490080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:43.490080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:43.490080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:43.490080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:43.490080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:43.490080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:43.490080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:43.490080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:43.490080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:22:43.490080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:43.490952 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf3425-97c8-404a-b6f9-028d64a709cf None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:43.491043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 516/1032] 10.222.0.64 () {56 vars in 1120 bytes} [Mon May 23 15:22:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:43.493691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfeb15-03b2-4bd2-b6cc-54158ef7302d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:43.529038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24df9a-ad92-426c-bb55-905189d40a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:43.529860 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24df9a-ad92-426c-bb55-905189d40a96 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:43.530158 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24df9a-ad92-426c-bb55-905189d40a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:43.530444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24df9a-ad92-426c-bb55-905189d40a96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:43.531273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24df9a-ad92-426c-bb55-905189d40a96 None admin] RBAC: Adding query filter params (interface=public, service_id=403f24222a724c298fe89eb1722f2add, region_id=RegionOne) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:43.532151 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24df9a-ad92-426c-bb55-905189d40a96 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:43.533551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24df9a-ad92-426c-bb55-905189d40a96 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:43.543245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 517/1033] 10.222.0.64 () {56 vars in 1251 bytes} [Mon May 23 15:22:43 2022] GET /identity/v3/endpoints?service_id=403f24222a724c298fe89eb1722f2add&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:43.544212 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24df9a-ad92-426c-bb55-905189d40a96 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:43.606579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfeb15-03b2-4bd2-b6cc-54158ef7302d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:43.636071 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:43.648784 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b4f0e-85b0-4d4b-aadd-4812ae0a27a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:43.662134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b4f0e-85b0-4d4b-aadd-4812ae0a27a3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:43.662134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b4f0e-85b0-4d4b-aadd-4812ae0a27a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:43.662134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b4f0e-85b0-4d4b-aadd-4812ae0a27a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:43.662134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b4f0e-85b0-4d4b-aadd-4812ae0a27a3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:43.662134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b4f0e-85b0-4d4b-aadd-4812ae0a27a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:43.662134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b4f0e-85b0-4d4b-aadd-4812ae0a27a3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:43.662624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24df9a-ad92-426c-bb55-905189d40a96 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:43.662711 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 517/1034] 10.222.0.64 () {56 vars in 1115 bytes} [Mon May 23 15:22:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:43.666670 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b4f0e-85b0-4d4b-aadd-4812ae0a27a3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:43.678272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:43.691521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81025620-9e9a-420b-95de-920ab929a78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:43.696365 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81025620-9e9a-420b-95de-920ab929a78e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:43.696365 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81025620-9e9a-420b-95de-920ab929a78e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:43.696365 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81025620-9e9a-420b-95de-920ab929a78e None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:43.696365 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81025620-9e9a-420b-95de-920ab929a78e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:43.696365 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81025620-9e9a-420b-95de-920ab929a78e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:43.705692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 518/1035] 10.222.0.64 () {56 vars in 1087 bytes} [Mon May 23 15:22:43 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:22:43.788381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b4f0e-85b0-4d4b-aadd-4812ae0a27a3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:43.825535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800601b-63c8-43d1-a953-b556da309668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:43.826355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800601b-63c8-43d1-a953-b556da309668 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:43.826639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800601b-63c8-43d1-a953-b556da309668 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:43.826916 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800601b-63c8-43d1-a953-b556da309668 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:43.838431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800601b-63c8-43d1-a953-b556da309668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:43.839891 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800601b-63c8-43d1-a953-b556da309668 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:43.859213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a800601b-63c8-43d1-a953-b556da309668 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:43.859213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:43.859213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:43.859213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:43.859213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:43.859213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:43.859213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:43.859213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:43.859213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:43.859213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:43.859213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:43.859213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:43.859213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:43.859213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:43.859213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:43.860604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:43.862228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:43.862228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:43.862228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:43.862228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:43.862228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 518/1036] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:43.866377 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81025620-9e9a-420b-95de-920ab929a78e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:43.973182 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81025620-9e9a-420b-95de-920ab929a78e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:44.006673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec9e3b-227e-4af4-a0f9-116406cd694c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:44.007352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec9e3b-227e-4af4-a0f9-116406cd694c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:44.007643 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec9e3b-227e-4af4-a0f9-116406cd694c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:44.007926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec9e3b-227e-4af4-a0f9-116406cd694c None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:44.008649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec9e3b-227e-4af4-a0f9-116406cd694c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:44.008962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec9e3b-227e-4af4-a0f9-116406cd694c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:44.010366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec9e3b-227e-4af4-a0f9-116406cd694c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:44.023047 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 519/1037] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:44.028446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800601b-63c8-43d1-a953-b556da309668 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:44.066828 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:44.132918 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800601b-63c8-43d1-a953-b556da309668 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:44.167409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb2d06-cf8b-48ca-bde2-6300e33ede83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:44.168209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb2d06-cf8b-48ca-bde2-6300e33ede83 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:44.168496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb2d06-cf8b-48ca-bde2-6300e33ede83 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:44.168776 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb2d06-cf8b-48ca-bde2-6300e33ede83 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:44.177100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb2d06-cf8b-48ca-bde2-6300e33ede83 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:44.178539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb2d06-cf8b-48ca-bde2-6300e33ede83 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-9abb2d06-cf8b-48ca-bde2-6300e33ede83 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:22:44.194022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:44.195066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:44.196068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:44.196068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:44.196068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:22:44.196068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:22:44.196068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:22:44.196068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:22:44.196068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:22:44.196068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:22:44.196068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:22:44.196068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:44.196068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 519/1038] 10.222.0.64 () {56 vars in 1093 bytes} [Mon May 23 15:22:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:44.202542 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec9e3b-227e-4af4-a0f9-116406cd694c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:44.257811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:44.320885 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec9e3b-227e-4af4-a0f9-116406cd694c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:44.354353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961ba7a-81e1-4c31-bae5-40f966ff1b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:44.355033 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961ba7a-81e1-4c31-bae5-40f966ff1b39 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:44.355324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961ba7a-81e1-4c31-bae5-40f966ff1b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:44.355605 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961ba7a-81e1-4c31-bae5-40f966ff1b39 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:44.356448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961ba7a-81e1-4c31-bae5-40f966ff1b39 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:44.356766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961ba7a-81e1-4c31-bae5-40f966ff1b39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:44.358264 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961ba7a-81e1-4c31-bae5-40f966ff1b39 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:44.381520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 520/1039] 10.222.0.64 () {56 vars in 1133 bytes} [Mon May 23 15:22:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:44.383047 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb2d06-cf8b-48ca-bde2-6300e33ede83 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:44.476334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:44.483028 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb2d06-cf8b-48ca-bde2-6300e33ede83 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:44.522389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9d408-94ae-46e6-ac39-08c19ab815f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:44.523047 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9d408-94ae-46e6-ac39-08c19ab815f8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:44.523331 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9d408-94ae-46e6-ac39-08c19ab815f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:44.523607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9d408-94ae-46e6-ac39-08c19ab815f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:44.590383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9d408-94ae-46e6-ac39-08c19ab815f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:44.591739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9d408-94ae-46e6-ac39-08c19ab815f8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:44.687083 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:44.696281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 520/1040] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:22:44 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:22:44.702195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961ba7a-81e1-4c31-bae5-40f966ff1b39 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:44.801784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961ba7a-81e1-4c31-bae5-40f966ff1b39 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:44.804096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:44.838015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14e985-1835-4276-96d7-ee272597b086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:44.838815 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14e985-1835-4276-96d7-ee272597b086 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:44.839099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14e985-1835-4276-96d7-ee272597b086 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:44.839375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14e985-1835-4276-96d7-ee272597b086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:44.850165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14e985-1835-4276-96d7-ee272597b086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:44.851557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14e985-1835-4276-96d7-ee272597b086 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:44.869754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-7c14e985-1835-4276-96d7-ee272597b086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:44.869754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:44.869754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:44.869754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:44.869754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:44.869754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:44.869754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:44.869754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:44.869754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:44.869754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:44.869754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:44.869754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:44.869754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:44.869754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:44.869754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:44.870738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:44.871691 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:44.871691 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:44.871691 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:44.871691 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:44.871691 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 521/1041] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:44.881106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9d408-94ae-46e6-ac39-08c19ab815f8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:44.981092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9d408-94ae-46e6-ac39-08c19ab815f8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:45.015257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5758f11-c90e-44e2-8a93-785ff95390c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:45.016484 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5758f11-c90e-44e2-8a93-785ff95390c2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:45.016801 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5758f11-c90e-44e2-8a93-785ff95390c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:45.017102 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5758f11-c90e-44e2-8a93-785ff95390c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:45.017825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5758f11-c90e-44e2-8a93-785ff95390c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:45.018148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5758f11-c90e-44e2-8a93-785ff95390c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:45.019565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5758f11-c90e-44e2-8a93-785ff95390c2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:45.032222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 521/1042] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:45.037667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14e985-1835-4276-96d7-ee272597b086 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:45.136876 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14e985-1835-4276-96d7-ee272597b086 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:45.174550 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8934a28-0180-4fb0-801a-18ff03a45359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:45.175367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8934a28-0180-4fb0-801a-18ff03a45359 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:45.175655 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8934a28-0180-4fb0-801a-18ff03a45359 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:45.176198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8934a28-0180-4fb0-801a-18ff03a45359 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:45.183493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8934a28-0180-4fb0-801a-18ff03a45359 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:45.184950 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8934a28-0180-4fb0-801a-18ff03a45359 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:45.198963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-d8934a28-0180-4fb0-801a-18ff03a45359 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:45.198963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:45.198963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:45.198963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:45.198963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:45.198963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:45.198963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:45.198963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:45.198963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:45.198963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:45.198963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:45.198963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:45.198963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:45.198963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:45.198963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:45.199902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:45.200836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:45.200836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:45.200836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:45.200836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:45.200836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:45.200836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:45.200836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:45.200836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:45.201913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 522/1043] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:45.206212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5758f11-c90e-44e2-8a93-785ff95390c2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:45.264659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:45.309011 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5758f11-c90e-44e2-8a93-785ff95390c2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:45.337689 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4831e7-d201-4934-9521-114f2b41e8f6 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:45.338012 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4831e7-d201-4934-9521-114f2b41e8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:45.338255 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4831e7-d201-4934-9521-114f2b41e8f6 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:45.339753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 523/1044] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:45.343275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d187d1-03c3-4b6e-998a-4c9159121c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:45.344767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d187d1-03c3-4b6e-998a-4c9159121c72 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:45.345193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d187d1-03c3-4b6e-998a-4c9159121c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:45.345486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d187d1-03c3-4b6e-998a-4c9159121c72 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:45.346062 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38eb1cc-18d6-4c30-8985-e7d5166ef85c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:45.346284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d187d1-03c3-4b6e-998a-4c9159121c72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:45.346416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38eb1cc-18d6-4c30-8985-e7d5166ef85c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:45.346593 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d187d1-03c3-4b6e-998a-4c9159121c72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:45.346689 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38eb1cc-18d6-4c30-8985-e7d5166ef85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:45.348030 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d187d1-03c3-4b6e-998a-4c9159121c72 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:45.362588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 522/1045] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:45.368378 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d187d1-03c3-4b6e-998a-4c9159121c72 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:45.372445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a38eb1cc-18d6-4c30-8985-e7d5166ef85c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:45.373692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a38eb1cc-18d6-4c30-8985-e7d5166ef85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:45.385474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:45.444175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 524/1046] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:45.467571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38eb1cc-18d6-4c30-8985-e7d5166ef85c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:45.475951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d187d1-03c3-4b6e-998a-4c9159121c72 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:45.493590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:45.512415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a28e0-7bc6-404f-adda-e58ab5d65f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:45.513160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a28e0-7bc6-404f-adda-e58ab5d65f81 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:45.513454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a28e0-7bc6-404f-adda-e58ab5d65f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:45.513847 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a28e0-7bc6-404f-adda-e58ab5d65f81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:45.514788 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a28e0-7bc6-404f-adda-e58ab5d65f81 None admin] RBAC: Adding query filter params (role.id=91ddccb742f94d19958d422cab370fa3, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=8da96bed73794fee8f719ec904936164) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:45.515102 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a28e0-7bc6-404f-adda-e58ab5d65f81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:45.516590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a28e0-7bc6-404f-adda-e58ab5d65f81 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:45.522778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 523/1047] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:22:45 2022] GET /identity/v3/role_assignments?user.id=8da96bed73794fee8f719ec904936164&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=91ddccb742f94d19958d422cab370fa3 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:45.576961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38eb1cc-18d6-4c30-8985-e7d5166ef85c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:45.615008 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d42c9c-8e55-4b58-974e-b80508055fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:45.615821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d42c9c-8e55-4b58-974e-b80508055fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:45.616106 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d42c9c-8e55-4b58-974e-b80508055fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:45.616382 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d42c9c-8e55-4b58-974e-b80508055fa9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:45.629328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d42c9c-8e55-4b58-974e-b80508055fa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c2d42c9c-8e55-4b58-974e-b80508055fa9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:45.637080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:45.643241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:45.644552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:45.644552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:45.644552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:45.644552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:45.644552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:45.644552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:45.644552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:45.644552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:45.644552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:45.644552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:45.644552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:45.644552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:45.644552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 525/1048] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:22:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:45.645461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a28e0-7bc6-404f-adda-e58ab5d65f81 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:45.761341 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a28e0-7bc6-404f-adda-e58ab5d65f81 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:45.798680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ec42a-ec4f-4275-8e27-e84fc180256e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:45.799368 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ec42a-ec4f-4275-8e27-e84fc180256e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:45.799659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ec42a-ec4f-4275-8e27-e84fc180256e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:45.799939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ec42a-ec4f-4275-8e27-e84fc180256e None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:45.800705 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ec42a-ec4f-4275-8e27-e84fc180256e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:45.802223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ec42a-ec4f-4275-8e27-e84fc180256e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:45.802696 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ec42a-ec4f-4275-8e27-e84fc180256e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:45.813629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 524/1049] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:22:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:45.819059 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d42c9c-8e55-4b58-974e-b80508055fa9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:45.829217 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:45.919835 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d42c9c-8e55-4b58-974e-b80508055fa9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:45.954085 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fe484-cf0d-41b8-8b6c-0452815ff1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:45.954895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fe484-cf0d-41b8-8b6c-0452815ff1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:45.955184 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fe484-cf0d-41b8-8b6c-0452815ff1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:45.955467 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fe484-cf0d-41b8-8b6c-0452815ff1c4 None admin] PATH_INFO: `/v3/users/e6c516f2c6ac4f7da35f850f8b438c7e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:45.964630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab6034-2a42-4fa8-8675-dabd6e915153 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:45.964919 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab6034-2a42-4fa8-8675-dabd6e915153 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:45.965236 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab6034-2a42-4fa8-8675-dabd6e915153 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:45.966861 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 525/1050] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:45.971791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fe484-cf0d-41b8-8b6c-0452815ff1c4 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c516f2c6ac4f7da35f850f8b438c7e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:45.972324 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266132a-b914-46c0-b62f-f7185cd5eea2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:45.972617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266132a-b914-46c0-b62f-f7185cd5eea2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:45.972896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266132a-b914-46c0-b62f-f7185cd5eea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:45.973291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fe484-cf0d-41b8-8b6c-0452815ff1c4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:45.975127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 526/1051] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:22:45 2022] GET /identity/v3/users/e6c516f2c6ac4f7da35f850f8b438c7e => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:45.980697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fe484-cf0d-41b8-8b6c-0452815ff1c4 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:45.993102 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3266132a-b914-46c0-b62f-f7185cd5eea2 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:45.994990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3266132a-b914-46c0-b62f-f7185cd5eea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:46.042154 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:46.047683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:46.075800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 526/1052] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:46.085396 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fe484-cf0d-41b8-8b6c-0452815ff1c4 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:46.099015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266132a-b914-46c0-b62f-f7185cd5eea2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:46.121335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d667895-f95e-4f20-83c3-44d419cfc25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:46.122231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d667895-f95e-4f20-83c3-44d419cfc25f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:46.122685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d667895-f95e-4f20-83c3-44d419cfc25f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:46.122963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d667895-f95e-4f20-83c3-44d419cfc25f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:46.191967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d667895-f95e-4f20-83c3-44d419cfc25f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:46.193461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d667895-f95e-4f20-83c3-44d419cfc25f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:46.197366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266132a-b914-46c0-b62f-f7185cd5eea2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:46.235524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87abf6dd-9c97-404f-ac96-74109c87a045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:46.236249 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abf6dd-9c97-404f-ac96-74109c87a045 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:46.236538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abf6dd-9c97-404f-ac96-74109c87a045 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:46.236820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abf6dd-9c97-404f-ac96-74109c87a045 None admin] PATH_INFO: `/v3/services/image` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:46.237517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87abf6dd-9c97-404f-ac96-74109c87a045 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:46.238893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87abf6dd-9c97-404f-ac96-74109c87a045 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-87abf6dd-9c97-404f-ac96-74109c87a045 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:46.252050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:46.253152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:46.258243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:46.258243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:46.258243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:46.258243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:46.258243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:46.258243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:46.258243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:46.258243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:46.258243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:46.258243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:46.258243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:22:46.258243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:22:46.258243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:22:46.258243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:46.258243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 527/1053] 10.222.0.64 () {56 vars in 1105 bytes} [Mon May 23 15:22:46 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:46.259819 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87abf6dd-9c97-404f-ac96-74109c87a045 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:46.269833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:46.298239 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:46.315798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 527/1054] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:22:45 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 15:22:46.321153 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d667895-f95e-4f20-83c3-44d419cfc25f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:46.378117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87abf6dd-9c97-404f-ac96-74109c87a045 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:46.419016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6164ef-1900-41e5-92cb-e8a7ab8efdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:46.419688 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6164ef-1900-41e5-92cb-e8a7ab8efdf2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:46.419978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6164ef-1900-41e5-92cb-e8a7ab8efdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:46.420269 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6164ef-1900-41e5-92cb-e8a7ab8efdf2 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:46.420985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6164ef-1900-41e5-92cb-e8a7ab8efdf2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:46.421325 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6164ef-1900-41e5-92cb-e8a7ab8efdf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:46.423249 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6164ef-1900-41e5-92cb-e8a7ab8efdf2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:46.433298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 528/1055] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:46.434152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6164ef-1900-41e5-92cb-e8a7ab8efdf2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:46.448302 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d667895-f95e-4f20-83c3-44d419cfc25f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:46.483418 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c2138-e97c-4203-8645-a61aa1daac8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:46.484865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c2138-e97c-4203-8645-a61aa1daac8b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:46.485190 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c2138-e97c-4203-8645-a61aa1daac8b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:46.485522 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c2138-e97c-4203-8645-a61aa1daac8b None admin] PATH_INFO: `/v3/projects/f2c2cb04ede1407ba33662be629abe00` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:46.486716 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c2138-e97c-4203-8645-a61aa1daac8b None admin] RBAC: Authorizing `identity:get_project(project_id=f2c2cb04ede1407ba33662be629abe00)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:46.488142 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c2138-e97c-4203-8645-a61aa1daac8b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:46.495210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 528/1056] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:22:46 2022] GET /identity/v3/projects/f2c2cb04ede1407ba33662be629abe00 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:46.496587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c2138-e97c-4203-8645-a61aa1daac8b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:46.529518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:46.544077 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:46.547571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6164ef-1900-41e5-92cb-e8a7ab8efdf2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:46.584499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42653bcb-d23e-4893-872d-d15724487da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:46.585194 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42653bcb-d23e-4893-872d-d15724487da1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:46.585481 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42653bcb-d23e-4893-872d-d15724487da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:46.585760 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42653bcb-d23e-4893-872d-d15724487da1 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:46.586478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42653bcb-d23e-4893-872d-d15724487da1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:46.586770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42653bcb-d23e-4893-872d-d15724487da1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:46.588236 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42653bcb-d23e-4893-872d-d15724487da1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:46.593293 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 529/1057] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:46.599558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42653bcb-d23e-4893-872d-d15724487da1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:46.603221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c2138-e97c-4203-8645-a61aa1daac8b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:46.639720 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65eece-880b-41de-b930-51a1307ebaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:46.640407 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65eece-880b-41de-b930-51a1307ebaf4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:46.641286 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65eece-880b-41de-b930-51a1307ebaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:46.641565 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65eece-880b-41de-b930-51a1307ebaf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:46.642397 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65eece-880b-41de-b930-51a1307ebaf4 None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=f2c2cb04ede1407ba33662be629abe00, user.id=e6c516f2c6ac4f7da35f850f8b438c7e) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:46.642707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65eece-880b-41de-b930-51a1307ebaf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:46.644141 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65eece-880b-41de-b930-51a1307ebaf4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:46.649700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 529/1058] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:22:46 2022] GET /identity/v3/role_assignments?user.id=e6c516f2c6ac4f7da35f850f8b438c7e&scope.project.id=f2c2cb04ede1407ba33662be629abe00&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:46.721755 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42653bcb-d23e-4893-872d-d15724487da1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:46.760680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db962e-cf52-4d53-9805-939fc08d2902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:46.761306 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db962e-cf52-4d53-9805-939fc08d2902 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:46.761598 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db962e-cf52-4d53-9805-939fc08d2902 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:46.761880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db962e-cf52-4d53-9805-939fc08d2902 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:46.762631 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db962e-cf52-4d53-9805-939fc08d2902 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:46.764019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db962e-cf52-4d53-9805-939fc08d2902 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:46.870709 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 530/1059] 10.222.0.64 () {60 vars in 1144 bytes} [Mon May 23 15:22:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:22:47.727869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ffa02-8e6a-4403-bc7a-74bb156e216d None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:47.728392 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ffa02-8e6a-4403-bc7a-74bb156e216d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:47.728473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ffa02-8e6a-4403-bc7a-74bb156e216d None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:47.729976 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 530/1060] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:47.735077 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfbe75-f7a6-4258-ac65-568432bc84f2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:47.735483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfbe75-f7a6-4258-ac65-568432bc84f2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:47.735652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfbe75-f7a6-4258-ac65-568432bc84f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:47.753769 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b0cfbe75-f7a6-4258-ac65-568432bc84f2 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:47.755005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b0cfbe75-f7a6-4258-ac65-568432bc84f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:47.770858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:47.833303 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 531/1061] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:47.855805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ffa02-8e6a-4403-bc7a-74bb156e216d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:47.948779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:47.955952 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ffa02-8e6a-4403-bc7a-74bb156e216d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:47.989325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de5105-bf72-4976-a3ee-7295cb284343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:47.990152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de5105-bf72-4976-a3ee-7295cb284343 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:47.990442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de5105-bf72-4976-a3ee-7295cb284343 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:47.990721 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de5105-bf72-4976-a3ee-7295cb284343 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:48.002934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de5105-bf72-4976-a3ee-7295cb284343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-52de5105-bf72-4976-a3ee-7295cb284343 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:48.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:48.012178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:48.013079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:48.013079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:48.013079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:48.013079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:48.013079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:48.013079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:48.013079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:48.013079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:48.013079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:48.013079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:48.013079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:22:48.013079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:48.017384 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 531/1062] 10.222.0.64 () {56 vars in 1120 bytes} [Mon May 23 15:22:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:48.018455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfbe75-f7a6-4258-ac65-568432bc84f2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:48.116482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfbe75-f7a6-4258-ac65-568432bc84f2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:48.150061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7e814-f028-4bd9-87cc-e84d0d2f581b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:48.150736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7e814-f028-4bd9-87cc-e84d0d2f581b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:48.151023 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7e814-f028-4bd9-87cc-e84d0d2f581b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:48.151301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7e814-f028-4bd9-87cc-e84d0d2f581b None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:48.152046 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7e814-f028-4bd9-87cc-e84d0d2f581b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:48.152355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7e814-f028-4bd9-87cc-e84d0d2f581b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:48.153764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7e814-f028-4bd9-87cc-e84d0d2f581b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:48.167017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 532/1063] 10.222.0.64 () {56 vars in 1115 bytes} [Mon May 23 15:22:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:48.168438 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de5105-bf72-4976-a3ee-7295cb284343 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:48.269561 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de5105-bf72-4976-a3ee-7295cb284343 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:48.303084 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649be901-47e1-4f95-9740-22d67cf241b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:48.303878 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649be901-47e1-4f95-9740-22d67cf241b4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:48.304160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649be901-47e1-4f95-9740-22d67cf241b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:48.304435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649be901-47e1-4f95-9740-22d67cf241b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:48.315094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649be901-47e1-4f95-9740-22d67cf241b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:48.316541 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649be901-47e1-4f95-9740-22d67cf241b4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:48.333426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-649be901-47e1-4f95-9740-22d67cf241b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:48.333426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:48.333426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:48.333426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:48.333426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:48.333426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:48.333426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:48.333426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:48.333426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:48.333426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:48.333426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:48.333426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:48.333426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:48.333426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:48.333426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:48.334362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:48.335310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:48.335310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:48.335310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:48.335310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:48.337374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 532/1064] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:48.340232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7e814-f028-4bd9-87cc-e84d0d2f581b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:48.352388 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:48.448905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7e814-f028-4bd9-87cc-e84d0d2f581b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:48.487264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f9ff5-f4e1-4823-aa4d-ea04f58f1102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:48.487934 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f9ff5-f4e1-4823-aa4d-ea04f58f1102 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:48.488222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f9ff5-f4e1-4823-aa4d-ea04f58f1102 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:48.488503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f9ff5-f4e1-4823-aa4d-ea04f58f1102 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:48.489253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f9ff5-f4e1-4823-aa4d-ea04f58f1102 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:48.489566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f9ff5-f4e1-4823-aa4d-ea04f58f1102 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:48.490929 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f9ff5-f4e1-4823-aa4d-ea04f58f1102 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:48.507257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 533/1065] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:48.508716 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649be901-47e1-4f95-9740-22d67cf241b4 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:48.521454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:48.625693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649be901-47e1-4f95-9740-22d67cf241b4 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:48.677431 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741947a-74c6-4fa3-94aa-a577c4998c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:48.678775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741947a-74c6-4fa3-94aa-a577c4998c06 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:48.679069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741947a-74c6-4fa3-94aa-a577c4998c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:48.679340 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741947a-74c6-4fa3-94aa-a577c4998c06 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:48.690715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741947a-74c6-4fa3-94aa-a577c4998c06 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:48.692279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741947a-74c6-4fa3-94aa-a577c4998c06 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-4741947a-74c6-4fa3-94aa-a577c4998c06 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:22:48.711772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:48.718175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:48.719934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:48.719934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:48.719934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:22:48.719934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:22:48.719934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:22:48.719934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:22:48.719934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:22:48.719934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:22:48.719934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:22:48.719934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:48.719934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 533/1066] 10.222.0.64 () {56 vars in 1093 bytes} [Mon May 23 15:22:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:48.721133 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f9ff5-f4e1-4823-aa4d-ea04f58f1102 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:48.805381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:48.845939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f9ff5-f4e1-4823-aa4d-ea04f58f1102 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:48.864582 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe3135-7a21-44aa-8c5d-36d0266b61e0 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:48.865137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe3135-7a21-44aa-8c5d-36d0266b61e0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:48.865260 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe3135-7a21-44aa-8c5d-36d0266b61e0 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:48.866686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 534/1067] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:48.872229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb243b-4411-4dc3-af9e-cc45482fcdc2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:48.872525 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb243b-4411-4dc3-af9e-cc45482fcdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:48.872808 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb243b-4411-4dc3-af9e-cc45482fcdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:48.892205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571addde-0378-4230-b280-ff23cd991beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:48.892901 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571addde-0378-4230-b280-ff23cd991beb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:48.893216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571addde-0378-4230-b280-ff23cd991beb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:48.893498 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571addde-0378-4230-b280-ff23cd991beb None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:48.894345 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571addde-0378-4230-b280-ff23cd991beb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:48.894659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571addde-0378-4230-b280-ff23cd991beb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:48.894963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2eeb243b-4411-4dc3-af9e-cc45482fcdc2 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:48.896087 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571addde-0378-4230-b280-ff23cd991beb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:48.896645 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2eeb243b-4411-4dc3-af9e-cc45482fcdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:48.919495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 534/1068] 10.222.0.64 () {56 vars in 1133 bytes} [Mon May 23 15:22:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:48.925444 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571addde-0378-4230-b280-ff23cd991beb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:48.975981 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:48.982749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 535/1069] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:49.017035 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb243b-4411-4dc3-af9e-cc45482fcdc2 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:49.042406 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571addde-0378-4230-b280-ff23cd991beb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:49.072113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:49.082662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20439ab4-437c-4a97-b384-5a88484d67a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:49.083471 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20439ab4-437c-4a97-b384-5a88484d67a0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:49.083791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20439ab4-437c-4a97-b384-5a88484d67a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:49.084028 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20439ab4-437c-4a97-b384-5a88484d67a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:49.094443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20439ab4-437c-4a97-b384-5a88484d67a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:49.095900 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20439ab4-437c-4a97-b384-5a88484d67a0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:49.115098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-20439ab4-437c-4a97-b384-5a88484d67a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:49.115098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:49.115098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:49.115098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:49.115098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:49.115098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:49.115098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:49.115098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:49.115098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:49.115098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:49.115098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:49.115098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:49.115098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:49.115098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:49.115098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:49.116093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:49.121878 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:49.121878 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:49.121878 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:49.121878 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:49.121878 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 535/1070] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:49.122177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20439ab4-437c-4a97-b384-5a88484d67a0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:49.136256 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb243b-4411-4dc3-af9e-cc45482fcdc2 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:49.177394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ec67e-9ce0-43c4-bf1d-0666d9c2a1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:49.178206 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ec67e-9ce0-43c4-bf1d-0666d9c2a1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:49.178486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ec67e-9ce0-43c4-bf1d-0666d9c2a1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:49.178765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ec67e-9ce0-43c4-bf1d-0666d9c2a1e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:49.189940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ec67e-9ce0-43c4-bf1d-0666d9c2a1e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-cb9ec67e-9ce0-43c4-bf1d-0666d9c2a1e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:49.198554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:49.200136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:49.201152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:49.201152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:49.201152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:49.201152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:49.201152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:49.201152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:49.201152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:49.201152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:49.201152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:49.201152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:49.201152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:49.201152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:49.201152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 536/1071] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:22:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:49.207309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e98c0e-ce31-45db-9495-87fe91e52d28 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:49.207804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e98c0e-ce31-45db-9495-87fe91e52d28 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:49.208256 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e98c0e-ce31-45db-9495-87fe91e52d28 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:49.209840 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 537/1072] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:49.214131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e98c0e-ce31-45db-9495-87fe91e52d28 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:49.238865 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20439ab4-437c-4a97-b384-5a88484d67a0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:49.277764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afae21-733e-4969-b776-7254962b0bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:49.278438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afae21-733e-4969-b776-7254962b0bbc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:49.278726 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afae21-733e-4969-b776-7254962b0bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:49.279002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afae21-733e-4969-b776-7254962b0bbc None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:49.279730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afae21-733e-4969-b776-7254962b0bbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:49.280049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afae21-733e-4969-b776-7254962b0bbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:49.280636 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:49.282027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afae21-733e-4969-b776-7254962b0bbc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:49.294456 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 536/1073] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:49.295837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d9fa8-eeb9-456c-a8f1-5f82d1476758 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:49.301995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d9fa8-eeb9-456c-a8f1-5f82d1476758 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:49.301995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d9fa8-eeb9-456c-a8f1-5f82d1476758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:49.316641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ec0d9fa8-eeb9-456c-a8f1-5f82d1476758 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:49.318637 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ec0d9fa8-eeb9-456c-a8f1-5f82d1476758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:49.325869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e98c0e-ce31-45db-9495-87fe91e52d28 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:49.352675 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:49.366865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be551844-7ae3-451d-bcc9-4d0190b62ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:49.367555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be551844-7ae3-451d-bcc9-4d0190b62ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:49.367849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be551844-7ae3-451d-bcc9-4d0190b62ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:49.368135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be551844-7ae3-451d-bcc9-4d0190b62ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:49.369543 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be551844-7ae3-451d-bcc9-4d0190b62ab8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:49.369543 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be551844-7ae3-451d-bcc9-4d0190b62ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:49.370610 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be551844-7ae3-451d-bcc9-4d0190b62ab8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:49.380469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 538/1074] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:22:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:49.381239 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be551844-7ae3-451d-bcc9-4d0190b62ab8 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:49.393048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 537/1075] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:49.397946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d9fa8-eeb9-456c-a8f1-5f82d1476758 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:49.481326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be551844-7ae3-451d-bcc9-4d0190b62ab8 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:49.500626 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:49.502093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d9fa8-eeb9-456c-a8f1-5f82d1476758 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:49.513890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ab4e6-00ce-4547-822b-de7a9729d175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:49.514675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ab4e6-00ce-4547-822b-de7a9729d175 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:49.514959 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ab4e6-00ce-4547-822b-de7a9729d175 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:49.515783 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ab4e6-00ce-4547-822b-de7a9729d175 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:49.522301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ab4e6-00ce-4547-822b-de7a9729d175 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:49.523731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ab4e6-00ce-4547-822b-de7a9729d175 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:49.535975 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145e336-35fe-46aa-89d9-bafcc7daa230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:49.536838 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145e336-35fe-46aa-89d9-bafcc7daa230 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:49.537147 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145e336-35fe-46aa-89d9-bafcc7daa230 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:49.537425 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145e336-35fe-46aa-89d9-bafcc7daa230 None admin] PATH_INFO: `/v3/users/e6c516f2c6ac4f7da35f850f8b438c7e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:49.538448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-993ab4e6-00ce-4547-822b-de7a9729d175 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:49.538448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:49.538448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:49.538448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:49.538448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:49.538448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:49.538448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:49.538448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:49.538448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:49.538448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:49.538448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:49.538448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:49.538448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:49.538448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:49.538448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:49.539489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:49.540519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:49.540519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:49.540519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:49.540519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:49.540519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:49.540519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:49.540519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:49.540519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:49.540967 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145e336-35fe-46aa-89d9-bafcc7daa230 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c516f2c6ac4f7da35f850f8b438c7e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:49.540967 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145e336-35fe-46aa-89d9-bafcc7daa230 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:49.546295 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 538/1076] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:22:49 2022] GET /identity/v3/users/e6c516f2c6ac4f7da35f850f8b438c7e => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:49.546295 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145e336-35fe-46aa-89d9-bafcc7daa230 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:49.546551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 539/1077] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:49.546551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ab4e6-00ce-4547-822b-de7a9729d175 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:49.576529 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:49.642417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145e336-35fe-46aa-89d9-bafcc7daa230 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:49.645002 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ab4e6-00ce-4547-822b-de7a9729d175 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:49.676619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb49825-e617-4af4-84c0-5ad138e879d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:49.677410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb49825-e617-4af4-84c0-5ad138e879d8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:49.677700 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb49825-e617-4af4-84c0-5ad138e879d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:49.677984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb49825-e617-4af4-84c0-5ad138e879d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:49.678188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9992efa-9092-4c50-b3b9-9c5011c3da13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:49.678850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9992efa-9092-4c50-b3b9-9c5011c3da13 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:49.679140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9992efa-9092-4c50-b3b9-9c5011c3da13 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:49.679423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9992efa-9092-4c50-b3b9-9c5011c3da13 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:49.680210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9992efa-9092-4c50-b3b9-9c5011c3da13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:49.680535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9992efa-9092-4c50-b3b9-9c5011c3da13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:49.682004 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9992efa-9092-4c50-b3b9-9c5011c3da13 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:49.694098 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 540/1078] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:49.694986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9992efa-9092-4c50-b3b9-9c5011c3da13 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:49.737842 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:49.752595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb49825-e617-4af4-84c0-5ad138e879d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:49.754123 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb49825-e617-4af4-84c0-5ad138e879d8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:49.787352 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:49.795967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9992efa-9092-4c50-b3b9-9c5011c3da13 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:49.830325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2fe792-d82c-4917-991f-e0e59a753220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:49.831114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fe792-d82c-4917-991f-e0e59a753220 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:49.831406 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fe792-d82c-4917-991f-e0e59a753220 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:49.831687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fe792-d82c-4917-991f-e0e59a753220 None admin] PATH_INFO: `/v3/projects/f2c2cb04ede1407ba33662be629abe00` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:49.832877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2fe792-d82c-4917-991f-e0e59a753220 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c2cb04ede1407ba33662be629abe00)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:49.834368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2fe792-d82c-4917-991f-e0e59a753220 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:49.836079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 541/1079] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:22:49 2022] GET /identity/v3/projects/f2c2cb04ede1407ba33662be629abe00 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:49.836825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2fe792-d82c-4917-991f-e0e59a753220 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:49.857219 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 539/1080] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:22:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:22:49.857219 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb49825-e617-4af4-84c0-5ad138e879d8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:49.935570 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2fe792-d82c-4917-991f-e0e59a753220 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:49.944653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:49.952745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb49825-e617-4af4-84c0-5ad138e879d8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:49.969086 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b9fa5-c94f-46b0-be68-dbcc5b9d9e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:49.969999 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b9fa5-c94f-46b0-be68-dbcc5b9d9e03 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:49.970329 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b9fa5-c94f-46b0-be68-dbcc5b9d9e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:49.970616 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b9fa5-c94f-46b0-be68-dbcc5b9d9e03 None admin] PATH_INFO: `/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/8da96bed73794fee8f719ec904936164/roles/91ddccb742f94d19958d422cab370fa3` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:49.972851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b9fa5-c94f-46b0-be68-dbcc5b9d9e03 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d81eafac474f37a71ee5cc292d172f, user_id=8da96bed73794fee8f719ec904936164, role_id=91ddccb742f94d19958d422cab370fa3)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:49.974343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b9fa5-c94f-46b0-be68-dbcc5b9d9e03 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:49.988201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2d113-b64d-4608-b01b-c55850c9c569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:49.989121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2d113-b64d-4608-b01b-c55850c9c569 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:49.989410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2d113-b64d-4608-b01b-c55850c9c569 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:49.989689 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2d113-b64d-4608-b01b-c55850c9c569 None admin] PATH_INFO: `/v3/projects/f2c2cb04ede1407ba33662be629abe00/users/e6c516f2c6ac4f7da35f850f8b438c7e/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:49.991797 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2d113-b64d-4608-b01b-c55850c9c569 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c2cb04ede1407ba33662be629abe00, user_id=e6c516f2c6ac4f7da35f850f8b438c7e, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:49.993251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2d113-b64d-4608-b01b-c55850c9c569 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:49.996264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:50.035006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 542/1081] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:22:49 2022] PUT /identity/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/8da96bed73794fee8f719ec904936164/roles/91ddccb742f94d19958d422cab370fa3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:22:50.035830 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b9fa5-c94f-46b0-be68-dbcc5b9d9e03 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:50.066588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 540/1082] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:22:49 2022] PUT /identity/v3/projects/f2c2cb04ede1407ba33662be629abe00/users/e6c516f2c6ac4f7da35f850f8b438c7e/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:22:50.154986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b9fa5-c94f-46b0-be68-dbcc5b9d9e03 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:50.195293 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fff892-5d60-4c9c-842c-045a67d2ee78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:50.196121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fff892-5d60-4c9c-842c-045a67d2ee78 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:50.196406 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fff892-5d60-4c9c-842c-045a67d2ee78 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:50.196682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fff892-5d60-4c9c-842c-045a67d2ee78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:50.209174 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fff892-5d60-4c9c-842c-045a67d2ee78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:50.210738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fff892-5d60-4c9c-842c-045a67d2ee78 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:50.230429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-36fff892-5d60-4c9c-842c-045a67d2ee78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:50.230429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:50.230429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:50.230429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:50.230429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:50.230429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:50.230429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:50.230429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:50.230429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:50.230429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:50.230429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:50.230429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:50.230429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:50.230429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:50.230429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:50.232274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:50.233291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:50.233291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:50.233291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:50.233291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:50.233291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 543/1083] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:50.242184 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2d113-b64d-4608-b01b-c55850c9c569 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:50.346957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2d113-b64d-4608-b01b-c55850c9c569 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:50.381647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90039c9-5e31-448b-9c75-0f1627927b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:50.382306 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90039c9-5e31-448b-9c75-0f1627927b4d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:50.382590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90039c9-5e31-448b-9c75-0f1627927b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:50.382866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90039c9-5e31-448b-9c75-0f1627927b4d None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:50.383580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90039c9-5e31-448b-9c75-0f1627927b4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:50.383888 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90039c9-5e31-448b-9c75-0f1627927b4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:50.385395 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90039c9-5e31-448b-9c75-0f1627927b4d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:50.397052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 541/1084] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:52.293113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed55e7e-3ecc-42a4-8d90-6b8345e56b73 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:52.293674 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed55e7e-3ecc-42a4-8d90-6b8345e56b73 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:52.293674 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed55e7e-3ecc-42a4-8d90-6b8345e56b73 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:52.295039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 544/1085] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:52.295180 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8ab8a-4cea-46e1-a91d-2ebd82cf7977 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:52.295552 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8ab8a-4cea-46e1-a91d-2ebd82cf7977 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:52.295732 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8ab8a-4cea-46e1-a91d-2ebd82cf7977 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:52.297432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 542/1086] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:22:52.299849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e0418-cd22-495f-b89a-071b80ac7b7b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:52.300137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e0418-cd22-495f-b89a-071b80ac7b7b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:52.300418 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e0418-cd22-495f-b89a-071b80ac7b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:52.302399 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a0895-e54c-4c21-aa52-1da56cc564d1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:52.302693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a0895-e54c-4c21-aa52-1da56cc564d1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:52.302991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a0895-e54c-4c21-aa52-1da56cc564d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:52.317420 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-623e0418-cd22-495f-b89a-071b80ac7b7b None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:52.318601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-623e0418-cd22-495f-b89a-071b80ac7b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:52.320740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a63a0895-e54c-4c21-aa52-1da56cc564d1 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:52.321931 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a63a0895-e54c-4c21-aa52-1da56cc564d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:52.332667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:52.372598 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:52.385031 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 545/1087] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:52.388704 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 543/1088] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:52.409443 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e0418-cd22-495f-b89a-071b80ac7b7b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:52.413710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a0895-e54c-4c21-aa52-1da56cc564d1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:52.507514 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e0418-cd22-495f-b89a-071b80ac7b7b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:52.519270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a0895-e54c-4c21-aa52-1da56cc564d1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:52.541563 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2676a-3172-424a-9353-17b129db1069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:52.542358 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2676a-3172-424a-9353-17b129db1069 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:52.542645 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2676a-3172-424a-9353-17b129db1069 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:52.542925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2676a-3172-424a-9353-17b129db1069 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:52.552914 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7aff71-b731-4a46-b1a1-de70eec982b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:52.553162 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2676a-3172-424a-9353-17b129db1069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:52.553698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7aff71-b731-4a46-b1a1-de70eec982b9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:52.553985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7aff71-b731-4a46-b1a1-de70eec982b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:52.554261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7aff71-b731-4a46-b1a1-de70eec982b9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-d6c2676a-3172-424a-9353-17b129db1069 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:52.560757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:52.561760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:52.568189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:52.568189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:52.568189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:52.568189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:52.568189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:52.568189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:52.568189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:52.568189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:52.568189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:52.568189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:52.568189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:22:52.568189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:52.568189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 546/1089] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:22:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:52.569076 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7aff71-b731-4a46-b1a1-de70eec982b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:22:52.569160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2676a-3172-424a-9353-17b129db1069 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-3c7aff71-b731-4a46-b1a1-de70eec982b9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:22:52.573017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:52.573972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:52.574834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:52.574834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:52.574834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:52.574834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:52.574834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:22:52.574834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:22:52.574834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:22:52.574834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:22:52.574834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:22:52.574834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:22:52.574834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:22:52.574834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:52.577213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 544/1090] 10.222.0.64 () {56 vars in 1120 bytes} [Mon May 23 15:22:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:52.580649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:52.581458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7aff71-b731-4a46-b1a1-de70eec982b9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:52.629193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:52.732434 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2676a-3172-424a-9353-17b129db1069 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:52.732655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7aff71-b731-4a46-b1a1-de70eec982b9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:52.766589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682d93c-84a9-4386-b13f-ae82992d1e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:52.766799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bf277-18cf-43f3-8c26-bfe36edc52d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:52.767262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682d93c-84a9-4386-b13f-ae82992d1e42 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:52.767375 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bf277-18cf-43f3-8c26-bfe36edc52d5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:52.767550 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682d93c-84a9-4386-b13f-ae82992d1e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:52.767667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bf277-18cf-43f3-8c26-bfe36edc52d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:52.767829 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682d93c-84a9-4386-b13f-ae82992d1e42 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:52.767933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bf277-18cf-43f3-8c26-bfe36edc52d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:52.768573 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682d93c-84a9-4386-b13f-ae82992d1e42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:52.768678 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bf277-18cf-43f3-8c26-bfe36edc52d5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:52.768886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bf277-18cf-43f3-8c26-bfe36edc52d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:52.768994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682d93c-84a9-4386-b13f-ae82992d1e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:52.770281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682d93c-84a9-4386-b13f-ae82992d1e42 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:52.770389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bf277-18cf-43f3-8c26-bfe36edc52d5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:52.778590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 545/1091] 10.222.0.64 () {56 vars in 1115 bytes} [Mon May 23 15:22:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:52.778734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 547/1092] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:22:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:52.784875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bf277-18cf-43f3-8c26-bfe36edc52d5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:52.785005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682d93c-84a9-4386-b13f-ae82992d1e42 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:52.854598 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:52.883183 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682d93c-84a9-4386-b13f-ae82992d1e42 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:52.884960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bf277-18cf-43f3-8c26-bfe36edc52d5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:52.893857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:52.917576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0472971-c05e-4fad-a7da-af8411f967e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:52.918370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0472971-c05e-4fad-a7da-af8411f967e9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:52.918655 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0472971-c05e-4fad-a7da-af8411f967e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:52.918938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0472971-c05e-4fad-a7da-af8411f967e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:52.919329 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cbc7b-6b58-4b2d-bba3-b30340343743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:52.920192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cbc7b-6b58-4b2d-bba3-b30340343743 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:52.920479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cbc7b-6b58-4b2d-bba3-b30340343743 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:52.920792 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cbc7b-6b58-4b2d-bba3-b30340343743 None admin] PATH_INFO: `/v3/users/e6c516f2c6ac4f7da35f850f8b438c7e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:52.922584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cbc7b-6b58-4b2d-bba3-b30340343743 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c516f2c6ac4f7da35f850f8b438c7e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:52.924008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cbc7b-6b58-4b2d-bba3-b30340343743 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:52.925668 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 546/1093] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:22:52 2022] GET /identity/v3/users/e6c516f2c6ac4f7da35f850f8b438c7e => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:52.930834 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0472971-c05e-4fad-a7da-af8411f967e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:52.931655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cbc7b-6b58-4b2d-bba3-b30340343743 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:52.935705 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0472971-c05e-4fad-a7da-af8411f967e9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:52.950003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c0472971-c05e-4fad-a7da-af8411f967e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:52.950003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:52.950003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:52.950003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:52.950003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:52.950003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:52.950003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:52.950003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:52.950003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:52.950003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:52.950003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:52.950003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:52.950003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:52.950003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:52.950003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:52.951104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:52.952069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:52.952069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:52.952069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:52.952069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:52.952069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 548/1094] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:52.958178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0472971-c05e-4fad-a7da-af8411f967e9 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:53.037088 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cbc7b-6b58-4b2d-bba3-b30340343743 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:53.059374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0472971-c05e-4fad-a7da-af8411f967e9 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:53.071210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34de06-d300-40e8-b33c-69b238144b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:53.071870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34de06-d300-40e8-b33c-69b238144b48 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:53.072159 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34de06-d300-40e8-b33c-69b238144b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:53.072440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34de06-d300-40e8-b33c-69b238144b48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:53.082607 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:53.092982 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383f3f6-248f-4101-9ac6-359914004e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:53.094322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383f3f6-248f-4101-9ac6-359914004e44 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:53.094612 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383f3f6-248f-4101-9ac6-359914004e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:53.095824 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383f3f6-248f-4101-9ac6-359914004e44 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:53.095824 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383f3f6-248f-4101-9ac6-359914004e44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:53.096111 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383f3f6-248f-4101-9ac6-359914004e44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:53.097547 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383f3f6-248f-4101-9ac6-359914004e44 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:53.109508 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 549/1095] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:53.115178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383f3f6-248f-4101-9ac6-359914004e44 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:53.118130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:53.139826 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34de06-d300-40e8-b33c-69b238144b48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:53.141974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34de06-d300-40e8-b33c-69b238144b48 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:53.212494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383f3f6-248f-4101-9ac6-359914004e44 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:53.240630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 547/1096] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:22:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 15:22:53.245672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2d9ae-1d44-44ca-86b5-7db05168f044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:53.246366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34de06-d300-40e8-b33c-69b238144b48 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:53.246467 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2d9ae-1d44-44ca-86b5-7db05168f044 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:53.246731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2d9ae-1d44-44ca-86b5-7db05168f044 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:53.247008 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2d9ae-1d44-44ca-86b5-7db05168f044 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:53.255115 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa2d9ae-1d44-44ca-86b5-7db05168f044 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:53.256566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa2d9ae-1d44-44ca-86b5-7db05168f044 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-9fa2d9ae-1d44-44ca-86b5-7db05168f044 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:22:53.271308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:53.272429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:53.273446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:53.273446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:53.273446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:22:53.273446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:22:53.273446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:22:53.273446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:22:53.273446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:22:53.273446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:22:53.273446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:22:53.273446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:22:53.273446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 550/1097] 10.222.0.64 () {56 vars in 1093 bytes} [Mon May 23 15:22:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:53.278007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2d9ae-1d44-44ca-86b5-7db05168f044 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:53.323671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:53.324749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:53.346215 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34de06-d300-40e8-b33c-69b238144b48 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:53.378209 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2d9ae-1d44-44ca-86b5-7db05168f044 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:53.379576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0c26b-08e1-4c6c-97fb-2c22452b00f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:53.380439 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0c26b-08e1-4c6c-97fb-2c22452b00f4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:53.380718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0c26b-08e1-4c6c-97fb-2c22452b00f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:53.380993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0c26b-08e1-4c6c-97fb-2c22452b00f4 None admin] PATH_INFO: `/v3/projects/f2c2cb04ede1407ba33662be629abe00` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:53.382242 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0c26b-08e1-4c6c-97fb-2c22452b00f4 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c2cb04ede1407ba33662be629abe00)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:53.383660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0c26b-08e1-4c6c-97fb-2c22452b00f4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:53.385531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 548/1098] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:22:53 2022] GET /identity/v3/projects/f2c2cb04ede1407ba33662be629abe00 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:53.392234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0c26b-08e1-4c6c-97fb-2c22452b00f4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:53.412287 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f529c-c1b0-44fe-bafa-75c1a24bd530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:53.412974 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f529c-c1b0-44fe-bafa-75c1a24bd530 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:53.413292 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f529c-c1b0-44fe-bafa-75c1a24bd530 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:53.413610 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f529c-c1b0-44fe-bafa-75c1a24bd530 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:53.414456 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f529c-c1b0-44fe-bafa-75c1a24bd530 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:53.414771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f529c-c1b0-44fe-bafa-75c1a24bd530 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:53.416224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f529c-c1b0-44fe-bafa-75c1a24bd530 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:53.433439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 551/1099] 10.222.0.64 () {56 vars in 1133 bytes} [Mon May 23 15:22:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:53.439761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f529c-c1b0-44fe-bafa-75c1a24bd530 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:53.491653 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0c26b-08e1-4c6c-97fb-2c22452b00f4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:53.526313 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9abc4-6bd7-4eea-93e2-9be49c526ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:53.526964 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9abc4-6bd7-4eea-93e2-9be49c526ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:53.527245 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9abc4-6bd7-4eea-93e2-9be49c526ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:53.527517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9abc4-6bd7-4eea-93e2-9be49c526ea2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:53.528356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9abc4-6bd7-4eea-93e2-9be49c526ea2 None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=f2c2cb04ede1407ba33662be629abe00, user.id=e6c516f2c6ac4f7da35f850f8b438c7e) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:53.528666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9abc4-6bd7-4eea-93e2-9be49c526ea2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:53.530144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9abc4-6bd7-4eea-93e2-9be49c526ea2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:53.545928 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 549/1100] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:22:53 2022] GET /identity/v3/role_assignments?user.id=e6c516f2c6ac4f7da35f850f8b438c7e&scope.project.id=f2c2cb04ede1407ba33662be629abe00&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:53.546190 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f529c-c1b0-44fe-bafa-75c1a24bd530 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:53.546190 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:53.579520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59627d7b-dee1-489f-8548-0667ce7ebf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:53.580201 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59627d7b-dee1-489f-8548-0667ce7ebf9f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:53.580491 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59627d7b-dee1-489f-8548-0667ce7ebf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:53.580773 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59627d7b-dee1-489f-8548-0667ce7ebf9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:53.655508 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59627d7b-dee1-489f-8548-0667ce7ebf9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:53.656943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59627d7b-dee1-489f-8548-0667ce7ebf9f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:53.760715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 552/1101] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:22:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 15:22:53.767379 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9abc4-6bd7-4eea-93e2-9be49c526ea2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:53.778206 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:53.863558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9abc4-6bd7-4eea-93e2-9be49c526ea2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:53.896895 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d87df-6bea-410e-bf0c-517fd78b736c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:53.897710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d87df-6bea-410e-bf0c-517fd78b736c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:53.897997 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d87df-6bea-410e-bf0c-517fd78b736c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:53.898270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d87df-6bea-410e-bf0c-517fd78b736c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:53.908777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d87df-6bea-410e-bf0c-517fd78b736c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:53.910196 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d87df-6bea-410e-bf0c-517fd78b736c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:53.927775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-798d87df-6bea-410e-bf0c-517fd78b736c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:53.927775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:53.927775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:53.927775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:53.927775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:53.927775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:53.927775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:53.927775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:53.927775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:53.927775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:53.927775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:53.927775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:53.927775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:22:53.927775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:22:53.927775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:53.928702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:53.929669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:22:53.929669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:22:53.929669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:22:53.929669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:53.929669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 550/1102] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:22:53.934593 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59627d7b-dee1-489f-8548-0667ce7ebf9f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:53.936759 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:54.034672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59627d7b-dee1-489f-8548-0667ce7ebf9f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:54.067668 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee9aed-1c4c-4aa2-8ff7-e5a78cb5e41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:54.068361 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee9aed-1c4c-4aa2-8ff7-e5a78cb5e41a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:54.068652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee9aed-1c4c-4aa2-8ff7-e5a78cb5e41a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:54.068931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee9aed-1c4c-4aa2-8ff7-e5a78cb5e41a None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:54.069676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee9aed-1c4c-4aa2-8ff7-e5a78cb5e41a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:54.069986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee9aed-1c4c-4aa2-8ff7-e5a78cb5e41a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:54.071344 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee9aed-1c4c-4aa2-8ff7-e5a78cb5e41a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:54.083384 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 553/1103] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:54.088984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d87df-6bea-410e-bf0c-517fd78b736c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:54.147056 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:54.186449 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d87df-6bea-410e-bf0c-517fd78b736c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:54.219697 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a242871-f8d7-4c1c-9d8c-703cda5b9e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:54.220490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a242871-f8d7-4c1c-9d8c-703cda5b9e5d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:54.220775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a242871-f8d7-4c1c-9d8c-703cda5b9e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:54.221047 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a242871-f8d7-4c1c-9d8c-703cda5b9e5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:54.227533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a242871-f8d7-4c1c-9d8c-703cda5b9e5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:54.228955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a242871-f8d7-4c1c-9d8c-703cda5b9e5d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:54.242377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-0a242871-f8d7-4c1c-9d8c-703cda5b9e5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:54.242377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:22:54.242377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:22:54.242377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:22:54.242377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:22:54.242377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:22:54.242377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:22:54.242377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:22:54.242377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:22:54.242377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:22:54.242377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:22:54.242377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:22:54.242377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:22:54.242377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:22:54.242377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:22:54.243452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:22:54.249800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:22:54.249800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:22:54.249800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:22:54.249800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:22:54.249800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:22:54.249800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:22:54.249800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:22:54.249800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:22:54.249800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 551/1104] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:22:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:22:54.250854 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee9aed-1c4c-4aa2-8ff7-e5a78cb5e41a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:54.310014 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:54.348458 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee9aed-1c4c-4aa2-8ff7-e5a78cb5e41a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:54.381990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5dd92-b16e-4369-80cf-1b30d456d520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:54.382653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5dd92-b16e-4369-80cf-1b30d456d520 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:54.382936 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5dd92-b16e-4369-80cf-1b30d456d520 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:54.383212 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5dd92-b16e-4369-80cf-1b30d456d520 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:54.384042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5dd92-b16e-4369-80cf-1b30d456d520 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:54.384354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5dd92-b16e-4369-80cf-1b30d456d520 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:54.385819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5dd92-b16e-4369-80cf-1b30d456d520 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:54.398546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 554/1105] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:22:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:22:54.404474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a242871-f8d7-4c1c-9d8c-703cda5b9e5d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:54.514134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a242871-f8d7-4c1c-9d8c-703cda5b9e5d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:54.534075 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:54.547602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668c967-0dbe-4c36-aec7-03f15cdddfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:54.548251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668c967-0dbe-4c36-aec7-03f15cdddfc7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:54.548534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668c967-0dbe-4c36-aec7-03f15cdddfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:54.548809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668c967-0dbe-4c36-aec7-03f15cdddfc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:54.549663 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668c967-0dbe-4c36-aec7-03f15cdddfc7 None admin] RBAC: Adding query filter params (role.id=91ddccb742f94d19958d422cab370fa3, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=8da96bed73794fee8f719ec904936164) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:54.549976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668c967-0dbe-4c36-aec7-03f15cdddfc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:54.551648 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668c967-0dbe-4c36-aec7-03f15cdddfc7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:54.557220 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 552/1106] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:22:54 2022] GET /identity/v3/role_assignments?user.id=8da96bed73794fee8f719ec904936164&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=91ddccb742f94d19958d422cab370fa3 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:55.784922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3f581-9d27-4899-9342-1c1174f16f77 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:55.785461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3f581-9d27-4899-9342-1c1174f16f77 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:55.785548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3f581-9d27-4899-9342-1c1174f16f77 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:55.786930 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 555/1107] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:55.792067 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6495d1a-5e9b-484b-ab06-699f5d026acf None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:55.792501 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6495d1a-5e9b-484b-ab06-699f5d026acf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:55.792604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6495d1a-5e9b-484b-ab06-699f5d026acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:55.809331 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b6495d1a-5e9b-484b-ab06-699f5d026acf None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:55.810498 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b6495d1a-5e9b-484b-ab06-699f5d026acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:55.874667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 553/1108] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:22:55.901040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3f581-9d27-4899-9342-1c1174f16f77 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:55.997637 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3f581-9d27-4899-9342-1c1174f16f77 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:56.019990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:56.030779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7d337-b314-4102-9d90-0c1a6083ddf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:56.031551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7d337-b314-4102-9d90-0c1a6083ddf0 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:56.036131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7d337-b314-4102-9d90-0c1a6083ddf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:56.036131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7d337-b314-4102-9d90-0c1a6083ddf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:56.036131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7d337-b314-4102-9d90-0c1a6083ddf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:56.036131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7d337-b314-4102-9d90-0c1a6083ddf0 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:56.036436 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 556/1109] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:22:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:56.042015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6495d1a-5e9b-484b-ab06-699f5d026acf None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:56.139209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6495d1a-5e9b-484b-ab06-699f5d026acf None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:56.153078 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:56.173424 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734da567-bbdf-4723-8e22-15c78bc72deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:56.174087 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734da567-bbdf-4723-8e22-15c78bc72deb None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:56.174373 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734da567-bbdf-4723-8e22-15c78bc72deb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:56.174652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734da567-bbdf-4723-8e22-15c78bc72deb None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:56.175350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734da567-bbdf-4723-8e22-15c78bc72deb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:56.177044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734da567-bbdf-4723-8e22-15c78bc72deb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:56.258091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 554/1110] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:22:56 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:22:58.471696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b5d0b-3bff-4b39-b206-6d95c59194b8 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:58.472238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b5d0b-3bff-4b39-b206-6d95c59194b8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:58.472238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b5d0b-3bff-4b39-b206-6d95c59194b8 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:58.473665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 557/1111] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:22:58.478814 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad5976-18b5-4a3a-90ca-299b748fb96e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:58.479198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad5976-18b5-4a3a-90ca-299b748fb96e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:58.479369 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad5976-18b5-4a3a-90ca-299b748fb96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:58.496507 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1fad5976-18b5-4a3a-90ca-299b748fb96e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:22:58.497729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1fad5976-18b5-4a3a-90ca-299b748fb96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:22:58.563665 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 555/1112] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:22:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:22:58.586340 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b5d0b-3bff-4b39-b206-6d95c59194b8 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:58.683062 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b5d0b-3bff-4b39-b206-6d95c59194b8 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:58.717477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eab38-cd42-4df4-8105-0a500edbf875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:58.718529 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eab38-cd42-4df4-8105-0a500edbf875 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:58.718825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eab38-cd42-4df4-8105-0a500edbf875 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:58.719111 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eab38-cd42-4df4-8105-0a500edbf875 None admin] PATH_INFO: `/v3/roles/1bac0428b2544e2cb16fad539f080c14` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:58.720589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7eab38-cd42-4df4-8105-0a500edbf875 None admin] RBAC: Authorizing `identity:get_role(role_id=1bac0428b2544e2cb16fad539f080c14)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:58.722050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7eab38-cd42-4df4-8105-0a500edbf875 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:58.723376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 558/1113] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:22:58 2022] GET /identity/v3/roles/1bac0428b2544e2cb16fad539f080c14 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:58.729048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad5976-18b5-4a3a-90ca-299b748fb96e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:58.820453 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:58.823208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad5976-18b5-4a3a-90ca-299b748fb96e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:58.855190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e6dba-6c89-467d-b3cd-0bbfe66efdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:58.856097 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e6dba-6c89-467d-b3cd-0bbfe66efdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:58.856434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e6dba-6c89-467d-b3cd-0bbfe66efdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:58.856719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e6dba-6c89-467d-b3cd-0bbfe66efdb9 None admin] PATH_INFO: `/v3/users/a42f695e869b407192b77a2cc796fe9d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:58.869850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e6dba-6c89-467d-b3cd-0bbfe66efdb9 None admin] RBAC: Authorizing `identity:get_user(user_id=a42f695e869b407192b77a2cc796fe9d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:58.871408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e6dba-6c89-467d-b3cd-0bbfe66efdb9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:58.873032 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 556/1114] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:22:58 2022] GET /identity/v3/users/a42f695e869b407192b77a2cc796fe9d => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:58.879633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eab38-cd42-4df4-8105-0a500edbf875 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:58.936153 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:58.977226 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eab38-cd42-4df4-8105-0a500edbf875 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:59.009272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4efc71-ea5f-4800-bd31-cd3dc0e7d756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:59.009916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4efc71-ea5f-4800-bd31-cd3dc0e7d756 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:59.010204 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4efc71-ea5f-4800-bd31-cd3dc0e7d756 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:59.010483 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4efc71-ea5f-4800-bd31-cd3dc0e7d756 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:59.074786 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4efc71-ea5f-4800-bd31-cd3dc0e7d756 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:59.076226 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4efc71-ea5f-4800-bd31-cd3dc0e7d756 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:59.129749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:22:59.170635 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 559/1115] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:22:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:22:59.177180 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e6dba-6c89-467d-b3cd-0bbfe66efdb9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:59.273756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e6dba-6c89-467d-b3cd-0bbfe66efdb9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:59.307316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4674872-fe41-4e46-ad4e-bea358f28628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:59.308168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4674872-fe41-4e46-ad4e-bea358f28628 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:59.308459 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4674872-fe41-4e46-ad4e-bea358f28628 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:59.308741 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4674872-fe41-4e46-ad4e-bea358f28628 None admin] PATH_INFO: `/v3/projects/f2c2cb04ede1407ba33662be629abe00` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:59.309945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4674872-fe41-4e46-ad4e-bea358f28628 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c2cb04ede1407ba33662be629abe00)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:59.311365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4674872-fe41-4e46-ad4e-bea358f28628 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:59.313111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 557/1116] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:22:59 2022] GET /identity/v3/projects/f2c2cb04ede1407ba33662be629abe00 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:22:59.319005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4efc71-ea5f-4800-bd31-cd3dc0e7d756 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:22:59.416269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4efc71-ea5f-4800-bd31-cd3dc0e7d756 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:22:59.449930 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e5e36-9a61-4447-b90b-d0f3a7d93cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:22:59.450580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e5e36-9a61-4447-b90b-d0f3a7d93cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:22:59.450867 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e5e36-9a61-4447-b90b-d0f3a7d93cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:22:59.451149 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e5e36-9a61-4447-b90b-d0f3a7d93cc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:22:59.451975 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e5e36-9a61-4447-b90b-d0f3a7d93cc4 None admin] RBAC: Adding query filter params (role.id=1bac0428b2544e2cb16fad539f080c14, scope.project.id=f2c2cb04ede1407ba33662be629abe00, user.id=a42f695e869b407192b77a2cc796fe9d) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:22:59.453391 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e5e36-9a61-4447-b90b-d0f3a7d93cc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:22:59.454825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e5e36-9a61-4447-b90b-d0f3a7d93cc4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:22:59.460081 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 560/1117] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:22:59 2022] GET /identity/v3/role_assignments?user.id=a42f695e869b407192b77a2cc796fe9d&scope.project.id=f2c2cb04ede1407ba33662be629abe00&role.id=1bac0428b2544e2cb16fad539f080c14 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:01.610901 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0ea9a-bc1d-44c2-862d-d74afccb5d25 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:01.611454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0ea9a-bc1d-44c2-862d-d74afccb5d25 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:01.611564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0ea9a-bc1d-44c2-862d-d74afccb5d25 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:01.612843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 558/1118] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:01.618232 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15779434-6b71-4e83-8670-a3e585285d9f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:01.618660 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15779434-6b71-4e83-8670-a3e585285d9f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:01.618805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15779434-6b71-4e83-8670-a3e585285d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:01.636349 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-15779434-6b71-4e83-8670-a3e585285d9f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:01.637556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-15779434-6b71-4e83-8670-a3e585285d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:01.653707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:01.704123 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 561/1119] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:23:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:23:01.726360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0ea9a-bc1d-44c2-862d-d74afccb5d25 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:01.753251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:01.823622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0ea9a-bc1d-44c2-862d-d74afccb5d25 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:01.855458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83f671-b74e-4bd4-8ce6-3b4e60449da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:01.856302 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83f671-b74e-4bd4-8ce6-3b4e60449da0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:01.856582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83f671-b74e-4bd4-8ce6-3b4e60449da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:01.856857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83f671-b74e-4bd4-8ce6-3b4e60449da0 None admin] PATH_INFO: `/v3/roles/1bac0428b2544e2cb16fad539f080c14` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:01.858531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83f671-b74e-4bd4-8ce6-3b4e60449da0 None admin] RBAC: Authorizing `identity:get_role(role_id=1bac0428b2544e2cb16fad539f080c14)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:01.859977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83f671-b74e-4bd4-8ce6-3b4e60449da0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:01.861394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 559/1120] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:23:01 2022] GET /identity/v3/roles/1bac0428b2544e2cb16fad539f080c14 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:01.867089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15779434-6b71-4e83-8670-a3e585285d9f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:01.962772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15779434-6b71-4e83-8670-a3e585285d9f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:01.994636 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c91e3-95ec-4319-a9a9-13308a59bf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:01.995494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c91e3-95ec-4319-a9a9-13308a59bf6c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:01.995779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c91e3-95ec-4319-a9a9-13308a59bf6c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:01.996057 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c91e3-95ec-4319-a9a9-13308a59bf6c None admin] PATH_INFO: `/v3/users/a42f695e869b407192b77a2cc796fe9d` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:01.997256 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c91e3-95ec-4319-a9a9-13308a59bf6c None admin] RBAC: Authorizing `identity:get_user(user_id=a42f695e869b407192b77a2cc796fe9d)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:01.998704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c91e3-95ec-4319-a9a9-13308a59bf6c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:02.000814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 562/1121] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:23:01 2022] GET /identity/v3/users/a42f695e869b407192b77a2cc796fe9d => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:02.006446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83f671-b74e-4bd4-8ce6-3b4e60449da0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:02.095034 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:02.101176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83f671-b74e-4bd4-8ce6-3b4e60449da0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:02.135062 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cfa86-1143-47b3-8fdd-8c1a4343f323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:02.135861 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cfa86-1143-47b3-8fdd-8c1a4343f323 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:02.136149 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cfa86-1143-47b3-8fdd-8c1a4343f323 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:02.136429 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cfa86-1143-47b3-8fdd-8c1a4343f323 None admin] PATH_INFO: `/v3/projects/f2c2cb04ede1407ba33662be629abe00` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:02.137621 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cfa86-1143-47b3-8fdd-8c1a4343f323 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c2cb04ede1407ba33662be629abe00)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:02.139041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cfa86-1143-47b3-8fdd-8c1a4343f323 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:02.141222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 560/1122] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:23:02 2022] GET /identity/v3/projects/f2c2cb04ede1407ba33662be629abe00 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:02.147059 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c91e3-95ec-4319-a9a9-13308a59bf6c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:02.167850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:02.242451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c91e3-95ec-4319-a9a9-13308a59bf6c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:02.274439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a600e8-033b-474e-be53-f941b7c4e7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:02.275307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a600e8-033b-474e-be53-f941b7c4e7d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:02.275591 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a600e8-033b-474e-be53-f941b7c4e7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:02.275867 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a600e8-033b-474e-be53-f941b7c4e7d7 None admin] PATH_INFO: `/v3/projects/f2c2cb04ede1407ba33662be629abe00/users/a42f695e869b407192b77a2cc796fe9d/roles/1bac0428b2544e2cb16fad539f080c14` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:02.278042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a600e8-033b-474e-be53-f941b7c4e7d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2c2cb04ede1407ba33662be629abe00, user_id=a42f695e869b407192b77a2cc796fe9d, role_id=1bac0428b2544e2cb16fad539f080c14)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:02.279461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a600e8-033b-474e-be53-f941b7c4e7d7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:02.353657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 563/1123] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:23:02 2022] PUT /identity/v3/projects/f2c2cb04ede1407ba33662be629abe00/users/a42f695e869b407192b77a2cc796fe9d/roles/1bac0428b2544e2cb16fad539f080c14 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:23:04.496769 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e8ec3-df13-4220-9009-6c10f694b353 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:04.497364 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e8ec3-df13-4220-9009-6c10f694b353 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:04.497364 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e8ec3-df13-4220-9009-6c10f694b353 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:04.499403 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 561/1124] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:23:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:04.504515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d760aa-cecf-41e9-82c3-ea29ed4616c4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:04.504913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d760aa-cecf-41e9-82c3-ea29ed4616c4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:04.505099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d760aa-cecf-41e9-82c3-ea29ed4616c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:04.521566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-00d760aa-cecf-41e9-82c3-ea29ed4616c4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:04.522745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-00d760aa-cecf-41e9-82c3-ea29ed4616c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:04.588139 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 564/1125] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:23:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:23:04.611885 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e8ec3-df13-4220-9009-6c10f694b353 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:04.710611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e8ec3-df13-4220-9009-6c10f694b353 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:04.744328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47939567-a7b2-483c-b0ae-186cbacfe599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:04.745212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47939567-a7b2-483c-b0ae-186cbacfe599 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:04.745505 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47939567-a7b2-483c-b0ae-186cbacfe599 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:04.745789 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47939567-a7b2-483c-b0ae-186cbacfe599 None admin] PATH_INFO: `/v3/roles/1bac0428b2544e2cb16fad539f080c14` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:04.747302 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47939567-a7b2-483c-b0ae-186cbacfe599 None admin] RBAC: Authorizing `identity:get_role(role_id=1bac0428b2544e2cb16fad539f080c14)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:04.748736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47939567-a7b2-483c-b0ae-186cbacfe599 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:04.750137 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 562/1126] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:23:04 2022] GET /identity/v3/roles/1bac0428b2544e2cb16fad539f080c14 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:04.755815 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d760aa-cecf-41e9-82c3-ea29ed4616c4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:04.770318 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:04.857591 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d760aa-cecf-41e9-82c3-ea29ed4616c4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:04.891117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45d31a-b68f-4613-a26f-61fa875ee1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:04.891984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45d31a-b68f-4613-a26f-61fa875ee1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:04.892261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45d31a-b68f-4613-a26f-61fa875ee1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:04.892546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45d31a-b68f-4613-a26f-61fa875ee1f8 None admin] PATH_INFO: `/v3/users/a42f695e869b407192b77a2cc796fe9d` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:04.893744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45d31a-b68f-4613-a26f-61fa875ee1f8 None admin] RBAC: Authorizing `identity:get_user(user_id=a42f695e869b407192b77a2cc796fe9d)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:04.895183 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45d31a-b68f-4613-a26f-61fa875ee1f8 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:04.896807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 565/1127] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:23:04 2022] GET /identity/v3/users/a42f695e869b407192b77a2cc796fe9d => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:04.902620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47939567-a7b2-483c-b0ae-186cbacfe599 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:04.933051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:05.000730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47939567-a7b2-483c-b0ae-186cbacfe599 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:05.034611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a87082-4ec2-4024-bb06-f3124a6ae752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:05.035356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a87082-4ec2-4024-bb06-f3124a6ae752 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:05.035646 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a87082-4ec2-4024-bb06-f3124a6ae752 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:05.035925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a87082-4ec2-4024-bb06-f3124a6ae752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:05.100582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a87082-4ec2-4024-bb06-f3124a6ae752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:05.102074 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a87082-4ec2-4024-bb06-f3124a6ae752 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:05.129928 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:05.197035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 563/1128] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:23:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:23:05.203060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45d31a-b68f-4613-a26f-61fa875ee1f8 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:05.276691 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:05.297248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45d31a-b68f-4613-a26f-61fa875ee1f8 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:05.329715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22df269-18cc-42ba-8cd0-cfddca4a3992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:05.330560 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22df269-18cc-42ba-8cd0-cfddca4a3992 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:05.331099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22df269-18cc-42ba-8cd0-cfddca4a3992 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:05.331589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22df269-18cc-42ba-8cd0-cfddca4a3992 None admin] PATH_INFO: `/v3/projects/f2c2cb04ede1407ba33662be629abe00` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:05.333156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22df269-18cc-42ba-8cd0-cfddca4a3992 None admin] RBAC: Authorizing `identity:get_project(project_id=f2c2cb04ede1407ba33662be629abe00)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:05.334583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22df269-18cc-42ba-8cd0-cfddca4a3992 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:05.336245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 566/1129] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:23:05 2022] GET /identity/v3/projects/f2c2cb04ede1407ba33662be629abe00 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:05.342152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a87082-4ec2-4024-bb06-f3124a6ae752 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:05.437719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a87082-4ec2-4024-bb06-f3124a6ae752 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:05.464403 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:05.470462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71115e41-6f2d-4284-99b5-017625a7e3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:05.471103 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71115e41-6f2d-4284-99b5-017625a7e3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:05.471386 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71115e41-6f2d-4284-99b5-017625a7e3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:05.471664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71115e41-6f2d-4284-99b5-017625a7e3a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:05.472479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71115e41-6f2d-4284-99b5-017625a7e3a4 None admin] RBAC: Adding query filter params (role.id=1bac0428b2544e2cb16fad539f080c14, scope.project.id=f2c2cb04ede1407ba33662be629abe00, user.id=a42f695e869b407192b77a2cc796fe9d) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:23:05.472791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71115e41-6f2d-4284-99b5-017625a7e3a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:05.474226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71115e41-6f2d-4284-99b5-017625a7e3a4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:05.479787 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 564/1130] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:23:05 2022] GET /identity/v3/role_assignments?user.id=a42f695e869b407192b77a2cc796fe9d&scope.project.id=f2c2cb04ede1407ba33662be629abe00&role.id=1bac0428b2544e2cb16fad539f080c14 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:07.682950 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d25074-5c6b-4259-848e-759f0a67693b None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:07.683525 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d25074-5c6b-4259-848e-759f0a67693b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:07.683525 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d25074-5c6b-4259-848e-759f0a67693b None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:07.684878 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 567/1131] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:07.690502 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f3eaf-054c-4782-8a32-7a1673e351c4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:07.690900 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f3eaf-054c-4782-8a32-7a1673e351c4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:07.691055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f3eaf-054c-4782-8a32-7a1673e351c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:07.708236 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ea2f3eaf-054c-4782-8a32-7a1673e351c4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:07.709462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ea2f3eaf-054c-4782-8a32-7a1673e351c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:07.773076 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 565/1132] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:23:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:23:07.795918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d25074-5c6b-4259-848e-759f0a67693b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:07.894874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d25074-5c6b-4259-848e-759f0a67693b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:07.926858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3048e-fb93-443e-9d18-a0d87bf17061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:07.927641 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3048e-fb93-443e-9d18-a0d87bf17061 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:07.927928 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3048e-fb93-443e-9d18-a0d87bf17061 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:07.928209 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3048e-fb93-443e-9d18-a0d87bf17061 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:07.929399 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3048e-fb93-443e-9d18-a0d87bf17061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:07.930846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3048e-fb93-443e-9d18-a0d87bf17061 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:07.932424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 568/1133] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:23:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:07.938624 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f3eaf-054c-4782-8a32-7a1673e351c4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:08.034313 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f3eaf-054c-4782-8a32-7a1673e351c4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:08.057311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:08.068356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f483c-1aa0-497b-beef-d0031cc0847b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:08.069012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f483c-1aa0-497b-beef-d0031cc0847b None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:08.069767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f483c-1aa0-497b-beef-d0031cc0847b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:08.069767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f483c-1aa0-497b-beef-d0031cc0847b None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:08.070339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f483c-1aa0-497b-beef-d0031cc0847b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:08.071756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f483c-1aa0-497b-beef-d0031cc0847b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:08.172553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 566/1134] 10.222.0.64 () {60 vars in 1141 bytes} [Mon May 23 15:23:07 2022] POST /identity/v3/projects => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:23:10.397394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099f4c5-ff68-4b61-bf15-6b01acbe4d8d None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:10.397947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099f4c5-ff68-4b61-bf15-6b01acbe4d8d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:10.397947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099f4c5-ff68-4b61-bf15-6b01acbe4d8d None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:10.399333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 569/1135] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:10.404841 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdfd59-cf42-484a-be03-523a663dceb4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:10.405281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdfd59-cf42-484a-be03-523a663dceb4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:10.405511 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdfd59-cf42-484a-be03-523a663dceb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:10.422469 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e7fdfd59-cf42-484a-be03-523a663dceb4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:10.423656 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e7fdfd59-cf42-484a-be03-523a663dceb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:10.488740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 567/1136] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:23:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:23:10.514061 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099f4c5-ff68-4b61-bf15-6b01acbe4d8d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:10.529273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:10.618209 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099f4c5-ff68-4b61-bf15-6b01acbe4d8d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:10.651115 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a72012-9ae5-4833-bb38-e27b23afeeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:10.651834 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a72012-9ae5-4833-bb38-e27b23afeeda None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:10.652120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a72012-9ae5-4833-bb38-e27b23afeeda None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:10.652400 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a72012-9ae5-4833-bb38-e27b23afeeda None admin] PATH_INFO: `/v3/domains/default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:10.653590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a72012-9ae5-4833-bb38-e27b23afeeda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:10.655712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a72012-9ae5-4833-bb38-e27b23afeeda None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:10.657336 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 570/1137] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:23:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:10.663207 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fdfd59-cf42-484a-be03-523a663dceb4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:10.762119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fdfd59-cf42-484a-be03-523a663dceb4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:10.769226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:10.796265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508795e3-d71d-4e89-b222-42ed4f904a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:10.796922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508795e3-d71d-4e89-b222-42ed4f904a5d None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:10.797238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508795e3-d71d-4e89-b222-42ed4f904a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:10.797516 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508795e3-d71d-4e89-b222-42ed4f904a5d None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:10.798213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508795e3-d71d-4e89-b222-42ed4f904a5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:10.799689 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508795e3-d71d-4e89-b222-42ed4f904a5d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:10.846224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 568/1138] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:23:10 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:23:13.047708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955a38a-ce35-4fa5-a772-d38afacf8f08 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:13.048220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955a38a-ce35-4fa5-a772-d38afacf8f08 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:13.048312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955a38a-ce35-4fa5-a772-d38afacf8f08 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:13.049670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 571/1139] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:13.055183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cd3bb-87d9-4d16-b2ae-2081114fb718 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:13.055582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cd3bb-87d9-4d16-b2ae-2081114fb718 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:13.055738 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cd3bb-87d9-4d16-b2ae-2081114fb718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:13.072449 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0e9cd3bb-87d9-4d16-b2ae-2081114fb718 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:13.073650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0e9cd3bb-87d9-4d16-b2ae-2081114fb718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:13.138304 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 569/1140] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:23:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:23:13.161977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955a38a-ce35-4fa5-a772-d38afacf8f08 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:13.241437 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:13.259820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955a38a-ce35-4fa5-a772-d38afacf8f08 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:13.293170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5dc4bc-7289-44f5-b2fd-d846ebd0a837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:13.293959 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5dc4bc-7289-44f5-b2fd-d846ebd0a837 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:13.294249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5dc4bc-7289-44f5-b2fd-d846ebd0a837 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:13.294529 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5dc4bc-7289-44f5-b2fd-d846ebd0a837 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:13.305218 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5dc4bc-7289-44f5-b2fd-d846ebd0a837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-1f5dc4bc-7289-44f5-b2fd-d846ebd0a837 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:23:13.313479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:23:13.314599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:23:13.315631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:23:13.315631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:23:13.315631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:23:13.315631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:23:13.315631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:23:13.315631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:23:13.315631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:23:13.315631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:23:13.315631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:23:13.315631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:23:13.315631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:23:13.315631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:23:13.315631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 572/1141] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:23:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:23:13.320471 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9cd3bb-87d9-4d16-b2ae-2081114fb718 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:13.408016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:13.416452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9cd3bb-87d9-4d16-b2ae-2081114fb718 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:13.453008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91b653-cbed-49f9-8670-a08589c81c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:13.453725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91b653-cbed-49f9-8670-a08589c81c30 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:13.454018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91b653-cbed-49f9-8670-a08589c81c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:13.454302 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91b653-cbed-49f9-8670-a08589c81c30 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:13.455209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91b653-cbed-49f9-8670-a08589c81c30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:23:13.455524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91b653-cbed-49f9-8670-a08589c81c30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:13.456980 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91b653-cbed-49f9-8670-a08589c81c30 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:13.466599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 570/1142] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:23:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:13.473120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5dc4bc-7289-44f5-b2fd-d846ebd0a837 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:13.577214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5dc4bc-7289-44f5-b2fd-d846ebd0a837 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:13.609409 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3ad9b-5254-4b2e-8ed8-c0026955e44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:13.610331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3ad9b-5254-4b2e-8ed8-c0026955e44b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:13.610621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3ad9b-5254-4b2e-8ed8-c0026955e44b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:13.610902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3ad9b-5254-4b2e-8ed8-c0026955e44b None admin] PATH_INFO: `/v3/users/53aebf9874dd4d26a2a2014d20d9eba7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:13.621845 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:13.624296 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3ad9b-5254-4b2e-8ed8-c0026955e44b None admin] RBAC: Authorizing `identity:get_user(user_id=53aebf9874dd4d26a2a2014d20d9eba7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:13.625787 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3ad9b-5254-4b2e-8ed8-c0026955e44b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:13.627481 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 573/1143] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:23:13 2022] GET /identity/v3/users/53aebf9874dd4d26a2a2014d20d9eba7 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:13.633395 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91b653-cbed-49f9-8670-a08589c81c30 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:13.742104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91b653-cbed-49f9-8670-a08589c81c30 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:13.773802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85474aa0-0281-4521-aa70-391fcec28d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:13.774450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85474aa0-0281-4521-aa70-391fcec28d9c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:13.774740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85474aa0-0281-4521-aa70-391fcec28d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:13.775022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85474aa0-0281-4521-aa70-391fcec28d9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:13.798749 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:13.840275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85474aa0-0281-4521-aa70-391fcec28d9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:13.841685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85474aa0-0281-4521-aa70-391fcec28d9c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:13.936615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 571/1144] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:23:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:23:13.942463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3ad9b-5254-4b2e-8ed8-c0026955e44b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:14.037869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3ad9b-5254-4b2e-8ed8-c0026955e44b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:14.071118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932eb53-912d-452f-984c-a91d2b33cf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:14.071967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932eb53-912d-452f-984c-a91d2b33cf08 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:14.072254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932eb53-912d-452f-984c-a91d2b33cf08 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:14.072532 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932eb53-912d-452f-984c-a91d2b33cf08 None admin] PATH_INFO: `/v3/projects/c2a313c104c94f8faa3317c999028030` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:14.073713 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932eb53-912d-452f-984c-a91d2b33cf08 None admin] RBAC: Authorizing `identity:get_project(project_id=c2a313c104c94f8faa3317c999028030)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:14.075144 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932eb53-912d-452f-984c-a91d2b33cf08 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:14.076885 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 574/1145] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:23:13 2022] GET /identity/v3/projects/c2a313c104c94f8faa3317c999028030 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:14.082851 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85474aa0-0281-4521-aa70-391fcec28d9c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:14.136678 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:14.180117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85474aa0-0281-4521-aa70-391fcec28d9c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:14.213309 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18522608-23c8-4b88-9d40-961a9d98619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:14.213955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18522608-23c8-4b88-9d40-961a9d98619e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:14.214242 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18522608-23c8-4b88-9d40-961a9d98619e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:14.214522 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18522608-23c8-4b88-9d40-961a9d98619e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:14.215342 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18522608-23c8-4b88-9d40-961a9d98619e None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=c2a313c104c94f8faa3317c999028030, user.id=53aebf9874dd4d26a2a2014d20d9eba7) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:23:14.215657 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18522608-23c8-4b88-9d40-961a9d98619e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:14.217619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18522608-23c8-4b88-9d40-961a9d98619e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:14.222882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 572/1146] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:23:14 2022] GET /identity/v3/role_assignments?user.id=53aebf9874dd4d26a2a2014d20d9eba7&scope.project.id=c2a313c104c94f8faa3317c999028030&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:16.385882 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0524d6-9e79-4ae4-92f6-f0fd37b70c64 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:16.386449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0524d6-9e79-4ae4-92f6-f0fd37b70c64 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:16.386449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0524d6-9e79-4ae4-92f6-f0fd37b70c64 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:16.387830 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 575/1147] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:16.393957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd683e-1f2c-4a41-b0a2-161ef99c314c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:16.394403 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd683e-1f2c-4a41-b0a2-161ef99c314c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:16.394504 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd683e-1f2c-4a41-b0a2-161ef99c314c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:16.411344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9bbd683e-1f2c-4a41-b0a2-161ef99c314c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:16.412519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9bbd683e-1f2c-4a41-b0a2-161ef99c314c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:16.476714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 573/1148] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:23:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:23:16.499205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0524d6-9e79-4ae4-92f6-f0fd37b70c64 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:16.564570 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:16.607107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0524d6-9e79-4ae4-92f6-f0fd37b70c64 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:16.640277 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e0566-4bc9-45d9-9639-e216000769bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:16.640996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e0566-4bc9-45d9-9639-e216000769bd None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:16.641440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e0566-4bc9-45d9-9639-e216000769bd None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:16.641725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e0566-4bc9-45d9-9639-e216000769bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:16.651440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e0566-4bc9-45d9-9639-e216000769bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-c95e0566-4bc9-45d9-9639-e216000769bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:23:16.659114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:23:16.660311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:23:16.661500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:23:16.661500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:23:16.661500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:23:16.661500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:23:16.661500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:23:16.661500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:23:16.661500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:23:16.661500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:23:16.661500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:23:16.661500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:23:16.661500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:23:16.661500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:23:16.661500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 576/1149] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:23:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:23:16.666229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd683e-1f2c-4a41-b0a2-161ef99c314c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:16.714383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:16.764817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd683e-1f2c-4a41-b0a2-161ef99c314c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:16.798437 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2b8de-e346-45fa-b03f-5135ee8a1fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:16.799111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2b8de-e346-45fa-b03f-5135ee8a1fda None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:16.799402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2b8de-e346-45fa-b03f-5135ee8a1fda None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:16.799683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2b8de-e346-45fa-b03f-5135ee8a1fda None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:16.800693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2b8de-e346-45fa-b03f-5135ee8a1fda None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:23:16.801007 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2b8de-e346-45fa-b03f-5135ee8a1fda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:16.802405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2b8de-e346-45fa-b03f-5135ee8a1fda None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:16.811497 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 574/1150] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:23:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:16.817369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e0566-4bc9-45d9-9639-e216000769bd None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:16.913882 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e0566-4bc9-45d9-9639-e216000769bd None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:16.939264 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:16.946570 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420099a1-8019-49f5-a33a-0e04e6531e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:16.951874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420099a1-8019-49f5-a33a-0e04e6531e03 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:16.951874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420099a1-8019-49f5-a33a-0e04e6531e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:16.951874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420099a1-8019-49f5-a33a-0e04e6531e03 None admin] PATH_INFO: `/v3/users/53aebf9874dd4d26a2a2014d20d9eba7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:16.951874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420099a1-8019-49f5-a33a-0e04e6531e03 None admin] RBAC: Authorizing `identity:get_user(user_id=53aebf9874dd4d26a2a2014d20d9eba7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:16.951874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420099a1-8019-49f5-a33a-0e04e6531e03 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:16.952875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 577/1151] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:23:16 2022] GET /identity/v3/users/53aebf9874dd4d26a2a2014d20d9eba7 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:16.958647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2b8de-e346-45fa-b03f-5135ee8a1fda None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:17.055252 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2b8de-e346-45fa-b03f-5135ee8a1fda None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:17.065289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:17.088026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2475dc-73b3-4822-832a-a48f4a752659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:17.088874 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2475dc-73b3-4822-832a-a48f4a752659 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:17.089272 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2475dc-73b3-4822-832a-a48f4a752659 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:17.089559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2475dc-73b3-4822-832a-a48f4a752659 None admin] PATH_INFO: `/v3/projects/c2a313c104c94f8faa3317c999028030` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:17.090731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2475dc-73b3-4822-832a-a48f4a752659 None admin] RBAC: Authorizing `identity:get_project(project_id=c2a313c104c94f8faa3317c999028030)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:17.092376 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2475dc-73b3-4822-832a-a48f4a752659 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:17.094096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 575/1152] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:23:16 2022] GET /identity/v3/projects/c2a313c104c94f8faa3317c999028030 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:17.100114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420099a1-8019-49f5-a33a-0e04e6531e03 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:17.196061 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420099a1-8019-49f5-a33a-0e04e6531e03 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:17.227607 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f5afc-6b54-4651-a055-2c7571435307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:17.228480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f5afc-6b54-4651-a055-2c7571435307 None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:17.228769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f5afc-6b54-4651-a055-2c7571435307 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:17.229049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f5afc-6b54-4651-a055-2c7571435307 None admin] PATH_INFO: `/v3/projects/c2a313c104c94f8faa3317c999028030/users/53aebf9874dd4d26a2a2014d20d9eba7/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:17.231111 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f5afc-6b54-4651-a055-2c7571435307 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2a313c104c94f8faa3317c999028030, user_id=53aebf9874dd4d26a2a2014d20d9eba7, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:17.233034 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f5afc-6b54-4651-a055-2c7571435307 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:17.318965 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 578/1153] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:23:17 2022] PUT /identity/v3/projects/c2a313c104c94f8faa3317c999028030/users/53aebf9874dd4d26a2a2014d20d9eba7/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:23:19.490549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff5bed-4435-43bb-82c4-7a32afdda6a3 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:19.491110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff5bed-4435-43bb-82c4-7a32afdda6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:19.491110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff5bed-4435-43bb-82c4-7a32afdda6a3 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:19.492492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 576/1154] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:19.497995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47435e4-0751-4a27-9714-4bda574b89df None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:19.498433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47435e4-0751-4a27-9714-4bda574b89df None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:19.498584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47435e4-0751-4a27-9714-4bda574b89df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:19.515714 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f47435e4-0751-4a27-9714-4bda574b89df None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:19.516906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f47435e4-0751-4a27-9714-4bda574b89df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:19.566062 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:19.582253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 579/1155] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:23:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:23:19.606271 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff5bed-4435-43bb-82c4-7a32afdda6a3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:19.706796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff5bed-4435-43bb-82c4-7a32afdda6a3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:19.738915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec1bac-53f5-4f12-a7b8-ed5a6546e5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:19.739703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec1bac-53f5-4f12-a7b8-ed5a6546e5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:19.739985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec1bac-53f5-4f12-a7b8-ed5a6546e5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:19.740257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec1bac-53f5-4f12-a7b8-ed5a6546e5d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:19.755543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec1bac-53f5-4f12-a7b8-ed5a6546e5d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-cdec1bac-53f5-4f12-a7b8-ed5a6546e5d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:23:19.763176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:23:19.764794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:23:19.765986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:23:19.765986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:23:19.765986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:23:19.765986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:23:19.765986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:23:19.765986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:23:19.765986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:23:19.765986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:23:19.765986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:23:19.765986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:23:19.765986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:23:19.765986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:23:19.765986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 577/1156] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:23:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:23:19.771318 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47435e4-0751-4a27-9714-4bda574b89df None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:19.866731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47435e4-0751-4a27-9714-4bda574b89df None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:19.898299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafe0a4-d73a-4fe4-91c4-d17ac8d1f96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:19.898951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafe0a4-d73a-4fe4-91c4-d17ac8d1f96c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:19.899235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafe0a4-d73a-4fe4-91c4-d17ac8d1f96c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:19.899511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafe0a4-d73a-4fe4-91c4-d17ac8d1f96c None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:19.900248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafe0a4-d73a-4fe4-91c4-d17ac8d1f96c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:23:19.900556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafe0a4-d73a-4fe4-91c4-d17ac8d1f96c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:19.902102 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafe0a4-d73a-4fe4-91c4-d17ac8d1f96c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:19.910371 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 580/1157] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:23:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:19.916137 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec1bac-53f5-4f12-a7b8-ed5a6546e5d5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:19.956717 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:20.016330 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec1bac-53f5-4f12-a7b8-ed5a6546e5d5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:20.048509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c37d4f-ccc6-43d8-a4cf-66fde2b6fdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:20.049392 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c37d4f-ccc6-43d8-a4cf-66fde2b6fdea None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:20.049677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c37d4f-ccc6-43d8-a4cf-66fde2b6fdea None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:20.050196 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c37d4f-ccc6-43d8-a4cf-66fde2b6fdea None admin] PATH_INFO: `/v3/users/53aebf9874dd4d26a2a2014d20d9eba7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:20.051369 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c37d4f-ccc6-43d8-a4cf-66fde2b6fdea None admin] RBAC: Authorizing `identity:get_user(user_id=53aebf9874dd4d26a2a2014d20d9eba7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:20.052778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c37d4f-ccc6-43d8-a4cf-66fde2b6fdea None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:20.054414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 578/1158] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:23:19 2022] GET /identity/v3/users/53aebf9874dd4d26a2a2014d20d9eba7 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:20.060141 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafe0a4-d73a-4fe4-91c4-d17ac8d1f96c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:20.109320 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:20.159794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafe0a4-d73a-4fe4-91c4-d17ac8d1f96c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:20.193137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738059c-49b3-4a29-be85-6ef9888117fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:20.193778 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738059c-49b3-4a29-be85-6ef9888117fa None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:20.194067 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738059c-49b3-4a29-be85-6ef9888117fa None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:20.194348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738059c-49b3-4a29-be85-6ef9888117fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:20.258432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738059c-49b3-4a29-be85-6ef9888117fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:20.259872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738059c-49b3-4a29-be85-6ef9888117fa None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:20.305160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:20.354085 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 581/1159] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:23:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:23:20.360515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c37d4f-ccc6-43d8-a4cf-66fde2b6fdea None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:20.463522 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c37d4f-ccc6-43d8-a4cf-66fde2b6fdea None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:20.471569 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:20.498770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c34fe-9c60-4af0-be6a-68cc755bb8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:20.499633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c34fe-9c60-4af0-be6a-68cc755bb8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:20.499919 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c34fe-9c60-4af0-be6a-68cc755bb8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:20.500198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c34fe-9c60-4af0-be6a-68cc755bb8f7 None admin] PATH_INFO: `/v3/projects/c2a313c104c94f8faa3317c999028030` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:20.501416 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c34fe-9c60-4af0-be6a-68cc755bb8f7 None admin] RBAC: Authorizing `identity:get_project(project_id=c2a313c104c94f8faa3317c999028030)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:20.503162 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c34fe-9c60-4af0-be6a-68cc755bb8f7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:20.505423 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 579/1160] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:23:20 2022] GET /identity/v3/projects/c2a313c104c94f8faa3317c999028030 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:20.511433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738059c-49b3-4a29-be85-6ef9888117fa None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:20.607319 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738059c-49b3-4a29-be85-6ef9888117fa None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:20.640531 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb70a65-0acb-4b61-a2aa-aa51bf12e28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:20.641206 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb70a65-0acb-4b61-a2aa-aa51bf12e28f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:20.641495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb70a65-0acb-4b61-a2aa-aa51bf12e28f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:20.641773 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb70a65-0acb-4b61-a2aa-aa51bf12e28f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:20.642590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb70a65-0acb-4b61-a2aa-aa51bf12e28f None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=c2a313c104c94f8faa3317c999028030, user.id=53aebf9874dd4d26a2a2014d20d9eba7) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:23:20.642900 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb70a65-0acb-4b61-a2aa-aa51bf12e28f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:20.644313 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb70a65-0acb-4b61-a2aa-aa51bf12e28f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:20.649904 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 582/1161] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:23:20 2022] GET /identity/v3/role_assignments?user.id=53aebf9874dd4d26a2a2014d20d9eba7&scope.project.id=c2a313c104c94f8faa3317c999028030&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:22.846518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33e6ee-8487-46cf-9d2c-40ef958546e8 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:22.847154 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33e6ee-8487-46cf-9d2c-40ef958546e8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:22.847154 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33e6ee-8487-46cf-9d2c-40ef958546e8 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:22.848425 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 580/1162] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:22.853713 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee803a-1c78-490f-b316-b8faf18b7e35 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:22.854090 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee803a-1c78-490f-b316-b8faf18b7e35 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:22.854280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee803a-1c78-490f-b316-b8faf18b7e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:22.871005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e6ee803a-1c78-490f-b316-b8faf18b7e35 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:22.872180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e6ee803a-1c78-490f-b316-b8faf18b7e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:22.879301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:22.938468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 583/1163] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:23:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:23:22.960462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33e6ee-8487-46cf-9d2c-40ef958546e8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:23.058057 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33e6ee-8487-46cf-9d2c-40ef958546e8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:23.091975 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942ec84-e558-4600-819f-7205ce1fed50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:23.092616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942ec84-e558-4600-819f-7205ce1fed50 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:23.092905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942ec84-e558-4600-819f-7205ce1fed50 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:23.093218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942ec84-e558-4600-819f-7205ce1fed50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:23.133216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:23.158199 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942ec84-e558-4600-819f-7205ce1fed50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:23.159679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942ec84-e558-4600-819f-7205ce1fed50 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:23.254449 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 581/1164] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:23:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:23:23.260362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee803a-1c78-490f-b316-b8faf18b7e35 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:23.355280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee803a-1c78-490f-b316-b8faf18b7e35 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:23.387422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7be39-460a-4ed4-89dd-daf957056c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:23.388770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7be39-460a-4ed4-89dd-daf957056c1d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:23.389167 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7be39-460a-4ed4-89dd-daf957056c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:23.389450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7be39-460a-4ed4-89dd-daf957056c1d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:23.395889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7be39-460a-4ed4-89dd-daf957056c1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:23.397358 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7be39-460a-4ed4-89dd-daf957056c1d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-0ce7be39-460a-4ed4-89dd-daf957056c1d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:23:23.411177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:23:23.412240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:23:23.413196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:23:23.413196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:23:23.413196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:23:23.413196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:23:23.413196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:23:23.413196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:23:23.413196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:23:23.413196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:23:23.413196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:23:23.413196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:23:23.413196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 23 15:23:23.413196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:23:23.413196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 15:23:23.413196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:23:23.413196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 584/1165] 10.222.0.64 () {56 vars in 1117 bytes} [Mon May 23 15:23:23 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:23:23.418147 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942ec84-e558-4600-819f-7205ce1fed50 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:23.489687 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:23.513718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942ec84-e558-4600-819f-7205ce1fed50 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:23.545588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b215089-d580-4b53-9937-16216ec02e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:23.546242 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b215089-d580-4b53-9937-16216ec02e25 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:23.546526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b215089-d580-4b53-9937-16216ec02e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:23.546802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b215089-d580-4b53-9937-16216ec02e25 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:23.547507 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b215089-d580-4b53-9937-16216ec02e25 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:23:23.547813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b215089-d580-4b53-9937-16216ec02e25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:23.549174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b215089-d580-4b53-9937-16216ec02e25 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:23.556258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 582/1166] 10.222.0.64 () {56 vars in 1115 bytes} [Mon May 23 15:23:23 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:25.730904 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e787767-6408-4887-ab02-47cd20a5f202 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:25.731343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e787767-6408-4887-ab02-47cd20a5f202 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:25.731453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e787767-6408-4887-ab02-47cd20a5f202 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:25.733603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 585/1167] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:23:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:25.738896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad963-fdac-44a3-811e-46ad735af0ac None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:25.739336 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad963-fdac-44a3-811e-46ad735af0ac None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:25.739470 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad963-fdac-44a3-811e-46ad735af0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:25.756630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c87ad963-fdac-44a3-811e-46ad735af0ac None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:25.757838 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c87ad963-fdac-44a3-811e-46ad735af0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:25.822808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 583/1168] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:23:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:23:25.845464 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e787767-6408-4887-ab02-47cd20a5f202 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:25.848958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:25.942245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e787767-6408-4887-ab02-47cd20a5f202 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:25.974233 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94021126-1436-4272-b8c6-9ac9579ddd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:25.974819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94021126-1436-4272-b8c6-9ac9579ddd56 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:25.975103 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94021126-1436-4272-b8c6-9ac9579ddd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:25.975379 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94021126-1436-4272-b8c6-9ac9579ddd56 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:25.976104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94021126-1436-4272-b8c6-9ac9579ddd56 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:25.977514 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94021126-1436-4272-b8c6-9ac9579ddd56 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:26.022641 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 586/1169] 10.222.0.64 () {60 vars in 1138 bytes} [Mon May 23 15:23:25 2022] POST /identity/v3/domains => generated 267 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:23:28.235267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27f960-9d94-4272-a87e-3d8263d7e8ff None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:28.235820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27f960-9d94-4272-a87e-3d8263d7e8ff None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:28.235820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27f960-9d94-4272-a87e-3d8263d7e8ff None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:28.237202 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 584/1170] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:28.242422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818f399-6645-4f2e-bb65-f6c24c25327a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:28.242861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818f399-6645-4f2e-bb65-f6c24c25327a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:28.242986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818f399-6645-4f2e-bb65-f6c24c25327a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:28.259709 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-9818f399-6645-4f2e-bb65-f6c24c25327a None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:28.260887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9818f399-6645-4f2e-bb65-f6c24c25327a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:28.316315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:28.326131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 587/1171] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:23:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:23:28.349531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27f960-9d94-4272-a87e-3d8263d7e8ff None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:28.414700 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:28.448489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27f960-9d94-4272-a87e-3d8263d7e8ff None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:28.480555 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe3661-d2a4-4f71-9dba-c1257c84a1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:28.482310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe3661-d2a4-4f71-9dba-c1257c84a1ac None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:28.482310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe3661-d2a4-4f71-9dba-c1257c84a1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:28.482517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe3661-d2a4-4f71-9dba-c1257c84a1ac None admin] PATH_INFO: `/v3/domains/b0b50c662d9d426886ab3153f15bea9e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:28.492813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe3661-d2a4-4f71-9dba-c1257c84a1ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=b0b50c662d9d426886ab3153f15bea9e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:28.494223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe3661-d2a4-4f71-9dba-c1257c84a1ac None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:28.495852 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 585/1172] 10.222.0.64 () {56 vars in 1183 bytes} [Mon May 23 15:23:28 2022] GET /identity/v3/domains/b0b50c662d9d426886ab3153f15bea9e => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:28.502125 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818f399-6645-4f2e-bb65-f6c24c25327a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:28.598375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818f399-6645-4f2e-bb65-f6c24c25327a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:28.631991 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38ddc4-804b-4539-aab1-e7d21c69e41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:28.632654 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38ddc4-804b-4539-aab1-e7d21c69e41a None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:28.632938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38ddc4-804b-4539-aab1-e7d21c69e41a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:28.633256 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38ddc4-804b-4539-aab1-e7d21c69e41a None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:28.633969 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38ddc4-804b-4539-aab1-e7d21c69e41a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:28.635413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38ddc4-804b-4539-aab1-e7d21c69e41a None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:28.714273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 588/1173] 10.222.0.64 () {60 vars in 1141 bytes} [Mon May 23 15:23:28 2022] POST /identity/v3/projects => generated 361 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:23:30.910887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99941a5b-06a4-4b69-8ec6-a14e0d7bd8bb None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:30.911431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99941a5b-06a4-4b69-8ec6-a14e0d7bd8bb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:30.911431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99941a5b-06a4-4b69-8ec6-a14e0d7bd8bb None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:30.912815 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 586/1174] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:30.918085 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86be89-0ad8-494b-8252-9c0e02c105e9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:30.918492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86be89-0ad8-494b-8252-9c0e02c105e9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:30.918656 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86be89-0ad8-494b-8252-9c0e02c105e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:30.936085 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-5c86be89-0ad8-494b-8252-9c0e02c105e9 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:30.937303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5c86be89-0ad8-494b-8252-9c0e02c105e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:30.976127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:31.001314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 589/1175] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:23:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:23:31.025980 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99941a5b-06a4-4b69-8ec6-a14e0d7bd8bb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:31.124688 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99941a5b-06a4-4b69-8ec6-a14e0d7bd8bb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:31.149648 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:31.157434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384c137-c634-49d6-ada3-f19a4eaa387f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:31.158876 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384c137-c634-49d6-ada3-f19a4eaa387f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:31.158876 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384c137-c634-49d6-ada3-f19a4eaa387f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:31.158876 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384c137-c634-49d6-ada3-f19a4eaa387f None admin] PATH_INFO: `/v3/domains/b0b50c662d9d426886ab3153f15bea9e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:31.160118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384c137-c634-49d6-ada3-f19a4eaa387f None admin] RBAC: Authorizing `identity:get_domain(domain_id=b0b50c662d9d426886ab3153f15bea9e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:31.161570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384c137-c634-49d6-ada3-f19a4eaa387f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:31.163207 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 587/1176] 10.222.0.64 () {56 vars in 1183 bytes} [Mon May 23 15:23:31 2022] GET /identity/v3/domains/b0b50c662d9d426886ab3153f15bea9e => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:31.169128 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86be89-0ad8-494b-8252-9c0e02c105e9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:31.264826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86be89-0ad8-494b-8252-9c0e02c105e9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:31.297658 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb187f6-4d3b-4d2a-bc74-28022365d456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:31.298316 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb187f6-4d3b-4d2a-bc74-28022365d456 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:31.298601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb187f6-4d3b-4d2a-bc74-28022365d456 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:31.298880 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb187f6-4d3b-4d2a-bc74-28022365d456 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:31.299588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb187f6-4d3b-4d2a-bc74-28022365d456 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:31.301289 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb187f6-4d3b-4d2a-bc74-28022365d456 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:31.393332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 590/1177] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:23:31 2022] POST /identity/v3/users => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:23:33.616085 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ca4f4-12a7-4949-904d-65bc258beebf None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:33.616622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ca4f4-12a7-4949-904d-65bc258beebf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:33.620943 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ca4f4-12a7-4949-904d-65bc258beebf None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:33.620943 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 588/1178] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:33.623338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c0d63-af1a-4731-bb1c-8d36b70a5c98 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:33.623740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c0d63-af1a-4731-bb1c-8d36b70a5c98 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:33.623893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c0d63-af1a-4731-bb1c-8d36b70a5c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:33.641046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-671c0d63-af1a-4731-bb1c-8d36b70a5c98 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:33.642300 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-671c0d63-af1a-4731-bb1c-8d36b70a5c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:33.665595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:33.707836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 591/1179] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:23:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:23:33.731939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ca4f4-12a7-4949-904d-65bc258beebf None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:33.830928 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ca4f4-12a7-4949-904d-65bc258beebf None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:33.863172 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195f4b3-d805-4a36-b5e6-6acb733f60cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:33.863953 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195f4b3-d805-4a36-b5e6-6acb733f60cd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:33.864243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195f4b3-d805-4a36-b5e6-6acb733f60cd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:33.864522 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195f4b3-d805-4a36-b5e6-6acb733f60cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:33.875501 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195f4b3-d805-4a36-b5e6-6acb733f60cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-1195f4b3-d805-4a36-b5e6-6acb733f60cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:23:33.882974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:23:33.884051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:23:33.884957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:23:33.884957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:23:33.884957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:23:33.884957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:23:33.884957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:23:33.884957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:23:33.884957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:23:33.884957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:23:33.884957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:23:33.884957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:23:33.884957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:23:33.884957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:23:33.884957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 589/1180] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:23:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:23:33.890734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c0d63-af1a-4731-bb1c-8d36b70a5c98 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:33.985541 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c0d63-af1a-4731-bb1c-8d36b70a5c98 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:34.018461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2626d-9403-4709-aaa1-3c09776b337f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:34.019116 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2626d-9403-4709-aaa1-3c09776b337f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:34.019404 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2626d-9403-4709-aaa1-3c09776b337f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:34.019684 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2626d-9403-4709-aaa1-3c09776b337f None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:34.020426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2626d-9403-4709-aaa1-3c09776b337f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:23:34.020735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2626d-9403-4709-aaa1-3c09776b337f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:34.022134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2626d-9403-4709-aaa1-3c09776b337f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:34.030317 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 592/1181] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:23:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:34.035955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195f4b3-d805-4a36-b5e6-6acb733f60cd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:34.090824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:34.132658 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195f4b3-d805-4a36-b5e6-6acb733f60cd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:34.165981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdaa02-aebe-490f-9727-f15eacf6dd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:34.166837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdaa02-aebe-490f-9727-f15eacf6dd90 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:34.167127 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdaa02-aebe-490f-9727-f15eacf6dd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:34.167409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdaa02-aebe-490f-9727-f15eacf6dd90 None admin] PATH_INFO: `/v3/users/7d1e7a9bbb6948ff8a6a3d4dcf28159c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:34.180335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cdaa02-aebe-490f-9727-f15eacf6dd90 None admin] RBAC: Authorizing `identity:get_user(user_id=7d1e7a9bbb6948ff8a6a3d4dcf28159c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:34.181799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cdaa02-aebe-490f-9727-f15eacf6dd90 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:34.183434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 590/1182] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:23:34 2022] GET /identity/v3/users/7d1e7a9bbb6948ff8a6a3d4dcf28159c => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:34.189711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2626d-9403-4709-aaa1-3c09776b337f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:34.214066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:34.287134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2626d-9403-4709-aaa1-3c09776b337f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:34.320194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239d2b3-fb74-49a8-9e02-93c7dc657247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:34.320828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239d2b3-fb74-49a8-9e02-93c7dc657247 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:34.321140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239d2b3-fb74-49a8-9e02-93c7dc657247 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:34.321423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239d2b3-fb74-49a8-9e02-93c7dc657247 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:34.385237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239d2b3-fb74-49a8-9e02-93c7dc657247 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:34.386618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239d2b3-fb74-49a8-9e02-93c7dc657247 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:34.411898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:34.484261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 593/1183] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:23:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:23:34.490159 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdaa02-aebe-490f-9727-f15eacf6dd90 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:34.585049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdaa02-aebe-490f-9727-f15eacf6dd90 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:34.598350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:34.617375 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22331a40-d838-4f98-9a99-19b52de16313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:34.618223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22331a40-d838-4f98-9a99-19b52de16313 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:34.625472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22331a40-d838-4f98-9a99-19b52de16313 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:34.625472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22331a40-d838-4f98-9a99-19b52de16313 None admin] PATH_INFO: `/v3/projects/6d43e10e045a47da817c7603a392d482` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:34.625472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22331a40-d838-4f98-9a99-19b52de16313 None admin] RBAC: Authorizing `identity:get_project(project_id=6d43e10e045a47da817c7603a392d482)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:34.625472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22331a40-d838-4f98-9a99-19b52de16313 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:34.625472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 591/1184] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:23:34 2022] GET /identity/v3/projects/6d43e10e045a47da817c7603a392d482 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:34.629168 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239d2b3-fb74-49a8-9e02-93c7dc657247 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:34.730993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239d2b3-fb74-49a8-9e02-93c7dc657247 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:34.752432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:34.762957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae0b57-a598-4a69-b229-39b0527476b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:34.763601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae0b57-a598-4a69-b229-39b0527476b9 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:34.763886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae0b57-a598-4a69-b229-39b0527476b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:34.764164 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae0b57-a598-4a69-b229-39b0527476b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:34.764985 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae0b57-a598-4a69-b229-39b0527476b9 None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=6d43e10e045a47da817c7603a392d482, user.id=7d1e7a9bbb6948ff8a6a3d4dcf28159c) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:23:34.765327 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae0b57-a598-4a69-b229-39b0527476b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:34.766757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae0b57-a598-4a69-b229-39b0527476b9 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:34.771978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 594/1185] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:23:34 2022] GET /identity/v3/role_assignments?user.id=7d1e7a9bbb6948ff8a6a3d4dcf28159c&scope.project.id=6d43e10e045a47da817c7603a392d482&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:36.907548 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22109186-8895-4e7b-8cc7-c78c63d34dd6 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:36.908070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22109186-8895-4e7b-8cc7-c78c63d34dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:36.908178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22109186-8895-4e7b-8cc7-c78c63d34dd6 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:36.909499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 592/1186] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:36.914735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7deeb0-39b1-4664-a8f2-652abb910bfb None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:36.915106 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7deeb0-39b1-4664-a8f2-652abb910bfb None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:36.915295 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7deeb0-39b1-4664-a8f2-652abb910bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:36.932292 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2e7deeb0-39b1-4664-a8f2-652abb910bfb None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:36.933501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2e7deeb0-39b1-4664-a8f2-652abb910bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:36.996936 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 595/1187] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:23:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:23:37.018973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22109186-8895-4e7b-8cc7-c78c63d34dd6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:37.116019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22109186-8895-4e7b-8cc7-c78c63d34dd6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:37.147253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34715a14-4fe5-442e-b889-6db03fb134e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:37.148033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34715a14-4fe5-442e-b889-6db03fb134e9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:37.148317 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34715a14-4fe5-442e-b889-6db03fb134e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:37.148592 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34715a14-4fe5-442e-b889-6db03fb134e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:37.158988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34715a14-4fe5-442e-b889-6db03fb134e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-34715a14-4fe5-442e-b889-6db03fb134e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:23:37.166572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:23:37.167761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:23:37.169455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:23:37.169455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:23:37.169455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:23:37.169455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:23:37.169455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:23:37.169455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:23:37.169455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:23:37.169455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:23:37.169455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:23:37.169455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:23:37.169455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:23:37.169455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:23:37.169455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 593/1188] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:23:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:23:37.174850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7deeb0-39b1-4664-a8f2-652abb910bfb None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:37.270028 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7deeb0-39b1-4664-a8f2-652abb910bfb None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:37.286212 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:37.303039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0757b-e60a-43ca-b9ca-3084418e402e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:37.306006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0757b-e60a-43ca-b9ca-3084418e402e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:37.306006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0757b-e60a-43ca-b9ca-3084418e402e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:37.306006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0757b-e60a-43ca-b9ca-3084418e402e None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:37.306006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0757b-e60a-43ca-b9ca-3084418e402e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:23:37.306006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0757b-e60a-43ca-b9ca-3084418e402e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:37.306735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0757b-e60a-43ca-b9ca-3084418e402e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:37.314789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 596/1189] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:23:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:37.320855 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34715a14-4fe5-442e-b889-6db03fb134e9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:37.363217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:37.417379 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34715a14-4fe5-442e-b889-6db03fb134e9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:37.448769 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd8a54-bcaf-46dc-a1b1-bacc3b3d1204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:37.450205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8a54-bcaf-46dc-a1b1-bacc3b3d1204 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:37.450485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8a54-bcaf-46dc-a1b1-bacc3b3d1204 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:37.450759 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8a54-bcaf-46dc-a1b1-bacc3b3d1204 None admin] PATH_INFO: `/v3/users/7d1e7a9bbb6948ff8a6a3d4dcf28159c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:37.451927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd8a54-bcaf-46dc-a1b1-bacc3b3d1204 None admin] RBAC: Authorizing `identity:get_user(user_id=7d1e7a9bbb6948ff8a6a3d4dcf28159c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:37.453372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd8a54-bcaf-46dc-a1b1-bacc3b3d1204 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:37.454967 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 594/1190] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:23:37 2022] GET /identity/v3/users/7d1e7a9bbb6948ff8a6a3d4dcf28159c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:37.460587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0757b-e60a-43ca-b9ca-3084418e402e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:37.558893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0757b-e60a-43ca-b9ca-3084418e402e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:37.592669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66718640-c7a0-4e1d-bbaf-1c978eb29f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:37.593551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66718640-c7a0-4e1d-bbaf-1c978eb29f23 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:37.593840 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66718640-c7a0-4e1d-bbaf-1c978eb29f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:37.594118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66718640-c7a0-4e1d-bbaf-1c978eb29f23 None admin] PATH_INFO: `/v3/projects/6d43e10e045a47da817c7603a392d482` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:37.595290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66718640-c7a0-4e1d-bbaf-1c978eb29f23 None admin] RBAC: Authorizing `identity:get_project(project_id=6d43e10e045a47da817c7603a392d482)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:37.596731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66718640-c7a0-4e1d-bbaf-1c978eb29f23 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:37.598444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 597/1191] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:23:37 2022] GET /identity/v3/projects/6d43e10e045a47da817c7603a392d482 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:37.604250 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd8a54-bcaf-46dc-a1b1-bacc3b3d1204 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:37.702047 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd8a54-bcaf-46dc-a1b1-bacc3b3d1204 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:37.712016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:37.735002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6426ae6-21f0-4682-aedf-c2bb6df3aef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:37.735881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6426ae6-21f0-4682-aedf-c2bb6df3aef7 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:37.736167 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6426ae6-21f0-4682-aedf-c2bb6df3aef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:37.736446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6426ae6-21f0-4682-aedf-c2bb6df3aef7 None admin] PATH_INFO: `/v3/projects/6d43e10e045a47da817c7603a392d482/users/7d1e7a9bbb6948ff8a6a3d4dcf28159c/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:37.738402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6426ae6-21f0-4682-aedf-c2bb6df3aef7 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d43e10e045a47da817c7603a392d482, user_id=7d1e7a9bbb6948ff8a6a3d4dcf28159c, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:37.739867 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6426ae6-21f0-4682-aedf-c2bb6df3aef7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:37.797390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 595/1192] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:23:37 2022] PUT /identity/v3/projects/6d43e10e045a47da817c7603a392d482/users/7d1e7a9bbb6948ff8a6a3d4dcf28159c/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:23:39.946048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c374e-5ce9-46a5-859d-7dd4dc9ef991 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:39.946571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c374e-5ce9-46a5-859d-7dd4dc9ef991 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:39.946685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c374e-5ce9-46a5-859d-7dd4dc9ef991 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:39.948058 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 598/1193] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:23:39.953258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee08ba1-304c-4fe2-bc3f-411cf2a754a5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:39.953639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee08ba1-304c-4fe2-bc3f-411cf2a754a5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:39.953820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee08ba1-304c-4fe2-bc3f-411cf2a754a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:39.970743 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3ee08ba1-304c-4fe2-bc3f-411cf2a754a5 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:23:39.971903 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3ee08ba1-304c-4fe2-bc3f-411cf2a754a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:23:40.034794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 596/1194] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:23:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:23:40.057891 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c374e-5ce9-46a5-859d-7dd4dc9ef991 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:40.103682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:40.156399 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c374e-5ce9-46a5-859d-7dd4dc9ef991 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:40.188860 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618c7a5-79ce-4d9f-9d6f-debb56327746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:40.189662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618c7a5-79ce-4d9f-9d6f-debb56327746 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:40.189944 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618c7a5-79ce-4d9f-9d6f-debb56327746 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:40.190303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618c7a5-79ce-4d9f-9d6f-debb56327746 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:40.200111 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618c7a5-79ce-4d9f-9d6f-debb56327746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-0618c7a5-79ce-4d9f-9d6f-debb56327746 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:23:40.207555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:23:40.208651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:23:40.209638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:23:40.209638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:23:40.209638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:23:40.209638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:23:40.209638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:23:40.209638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:23:40.209638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:23:40.209638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:23:40.209638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:23:40.209638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:23:40.209638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:23:40.209638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:23:40.209638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 599/1195] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:23:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:23:40.214897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee08ba1-304c-4fe2-bc3f-411cf2a754a5 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:40.307877 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:40.310769 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee08ba1-304c-4fe2-bc3f-411cf2a754a5 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:40.343936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0f5f6-7224-499b-8cfd-d08fa0e3e6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:40.344601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0f5f6-7224-499b-8cfd-d08fa0e3e6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:40.344894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0f5f6-7224-499b-8cfd-d08fa0e3e6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:40.345212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0f5f6-7224-499b-8cfd-d08fa0e3e6f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:40.345956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0f5f6-7224-499b-8cfd-d08fa0e3e6f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:23:40.346271 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0f5f6-7224-499b-8cfd-d08fa0e3e6f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:40.347635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0f5f6-7224-499b-8cfd-d08fa0e3e6f9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:40.355385 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 597/1196] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:23:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:40.361628 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618c7a5-79ce-4d9f-9d6f-debb56327746 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:40.462498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618c7a5-79ce-4d9f-9d6f-debb56327746 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:40.470198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:40.508389 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59c4ba-c56a-4eed-8b6a-6ade879d6dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:40.509295 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59c4ba-c56a-4eed-8b6a-6ade879d6dda None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:40.509585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59c4ba-c56a-4eed-8b6a-6ade879d6dda None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:40.509868 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59c4ba-c56a-4eed-8b6a-6ade879d6dda None admin] PATH_INFO: `/v3/users/7d1e7a9bbb6948ff8a6a3d4dcf28159c` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:40.511050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59c4ba-c56a-4eed-8b6a-6ade879d6dda None admin] RBAC: Authorizing `identity:get_user(user_id=7d1e7a9bbb6948ff8a6a3d4dcf28159c)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:40.513660 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59c4ba-c56a-4eed-8b6a-6ade879d6dda None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:40.515292 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 600/1197] 10.222.0.64 () {56 vars in 1177 bytes} [Mon May 23 15:23:40 2022] GET /identity/v3/users/7d1e7a9bbb6948ff8a6a3d4dcf28159c => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:40.521082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0f5f6-7224-499b-8cfd-d08fa0e3e6f9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:40.623552 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0f5f6-7224-499b-8cfd-d08fa0e3e6f9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:40.656721 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a96bc-c197-4242-bb7e-e99a82c01e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:40.657391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a96bc-c197-4242-bb7e-e99a82c01e06 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:40.657680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a96bc-c197-4242-bb7e-e99a82c01e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:40.658101 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a96bc-c197-4242-bb7e-e99a82c01e06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:40.692907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:40.730373 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a96bc-c197-4242-bb7e-e99a82c01e06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:40.731935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a96bc-c197-4242-bb7e-e99a82c01e06 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:40.828928 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 598/1198] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:23:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:23:40.834807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59c4ba-c56a-4eed-8b6a-6ade879d6dda None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:40.932474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59c4ba-c56a-4eed-8b6a-6ade879d6dda None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:40.966112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7d2b1-24b9-4cb9-acd8-1642ca2c7506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:40.966962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7d2b1-24b9-4cb9-acd8-1642ca2c7506 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:40.967252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7d2b1-24b9-4cb9-acd8-1642ca2c7506 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:40.967533 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7d2b1-24b9-4cb9-acd8-1642ca2c7506 None admin] PATH_INFO: `/v3/projects/6d43e10e045a47da817c7603a392d482` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:40.968712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7d2b1-24b9-4cb9-acd8-1642ca2c7506 None admin] RBAC: Authorizing `identity:get_project(project_id=6d43e10e045a47da817c7603a392d482)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:40.970194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7d2b1-24b9-4cb9-acd8-1642ca2c7506 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:40.971883 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 601/1199] 10.222.0.64 () {56 vars in 1186 bytes} [Mon May 23 15:23:40 2022] GET /identity/v3/projects/6d43e10e045a47da817c7603a392d482 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:40.977936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a96bc-c197-4242-bb7e-e99a82c01e06 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:23:41.036438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:41.073803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a96bc-c197-4242-bb7e-e99a82c01e06 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:23:41.106931 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a35d9-8b05-4598-af20-94610d4cb1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:23:41.107569 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a35d9-8b05-4598-af20-94610d4cb1ca None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:23:41.107849 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a35d9-8b05-4598-af20-94610d4cb1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:23:41.108121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a35d9-8b05-4598-af20-94610d4cb1ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:23:41.108936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a35d9-8b05-4598-af20-94610d4cb1ca None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=6d43e10e045a47da817c7603a392d482, user.id=7d1e7a9bbb6948ff8a6a3d4dcf28159c) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:23:41.109276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a35d9-8b05-4598-af20-94610d4cb1ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:23:41.110683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a35d9-8b05-4598-af20-94610d4cb1ca None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:23:41.116226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 599/1200] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:23:40 2022] GET /identity/v3/role_assignments?user.id=7d1e7a9bbb6948ff8a6a3d4dcf28159c&scope.project.id=6d43e10e045a47da817c7603a392d482&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:23:50.962090 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=79 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:23:51.113694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=37 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:24:58.621332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4f110-1ee5-4c06-8bc0-84b2b0db37db None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:58.621930 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4f110-1ee5-4c06-8bc0-84b2b0db37db None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:58.621930 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4f110-1ee5-4c06-8bc0-84b2b0db37db None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:58.623293 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 602/1201] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:24:58.628303 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b6a3e-d303-46c7-82ac-9009e268908e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:58.628707 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b6a3e-d303-46c7-82ac-9009e268908e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:58.628863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b6a3e-d303-46c7-82ac-9009e268908e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:58.648542 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e63b6a3e-d303-46c7-82ac-9009e268908e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:24:58.649756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e63b6a3e-d303-46c7-82ac-9009e268908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:24:58.715986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 600/1202] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:24:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:24:58.740867 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4f110-1ee5-4c06-8bc0-84b2b0db37db None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:24:58.844453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4f110-1ee5-4c06-8bc0-84b2b0db37db None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:24:58.879697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab14cf-283e-4153-a34c-c3e3ed50a348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:24:58.880483 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab14cf-283e-4153-a34c-c3e3ed50a348 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:58.880776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab14cf-283e-4153-a34c-c3e3ed50a348 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:58.881084 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab14cf-283e-4153-a34c-c3e3ed50a348 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:58.891673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab14cf-283e-4153-a34c-c3e3ed50a348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:24:58.897342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab14cf-283e-4153-a34c-c3e3ed50a348 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:24:58.910290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-83ab14cf-283e-4153-a34c-c3e3ed50a348 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:24:58.910290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:24:58.910290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:24:58.910290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:24:58.910290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:24:58.910290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:24:58.910290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:24:58.910290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:24:58.910290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:24:58.910290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:24:58.910290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:24:58.910290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:24:58.910290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:24:58.910290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:24:58.910290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:24:58.911460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:24:58.912667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:24:58.912667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:24:58.912667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:24:58.912667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:24:58.912667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 603/1203] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:24:58.917543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b6a3e-d303-46c7-82ac-9009e268908e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:24:59.014619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b6a3e-d303-46c7-82ac-9009e268908e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:24:59.042295 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:24:59.047913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d900b-2aca-43d3-a801-02b7793f8f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:24:59.048825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d900b-2aca-43d3-a801-02b7793f8f43 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:59.049137 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d900b-2aca-43d3-a801-02b7793f8f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:59.049418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d900b-2aca-43d3-a801-02b7793f8f43 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:59.050130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d900b-2aca-43d3-a801-02b7793f8f43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:24:59.050437 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d900b-2aca-43d3-a801-02b7793f8f43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:24:59.051948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d900b-2aca-43d3-a801-02b7793f8f43 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:24:59.063307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 601/1204] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:24:59.069201 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab14cf-283e-4153-a34c-c3e3ed50a348 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:24:59.122733 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:24:59.169321 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab14cf-283e-4153-a34c-c3e3ed50a348 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:24:59.202239 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997200d-2477-4156-bffa-49d2a5e863c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:24:59.202912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997200d-2477-4156-bffa-49d2a5e863c7 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:24:59.203210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997200d-2477-4156-bffa-49d2a5e863c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:24:59.203493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997200d-2477-4156-bffa-49d2a5e863c7 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:24:59.204218 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997200d-2477-4156-bffa-49d2a5e863c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:24:59.205685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997200d-2477-4156-bffa-49d2a5e863c7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:24:59.304662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 604/1205] 10.222.0.64 () {60 vars in 1132 bytes} [Mon May 23 15:24:59 2022] POST /identity/v3/users => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:25:01.542598 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add19fb-e6a7-4e62-9176-7664e13dc53c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:01.543176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add19fb-e6a7-4e62-9176-7664e13dc53c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:01.543176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add19fb-e6a7-4e62-9176-7664e13dc53c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:01.544691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 602/1206] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:25:01.550188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13e029-da06-42c0-a179-5fb3594257af None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:01.550579 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13e029-da06-42c0-a179-5fb3594257af None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:01.556877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13e029-da06-42c0-a179-5fb3594257af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:01.567127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3d13e029-da06-42c0-a179-5fb3594257af None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:25:01.568315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3d13e029-da06-42c0-a179-5fb3594257af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:01.633827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 605/1207] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:25:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:25:01.657957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add19fb-e6a7-4e62-9176-7664e13dc53c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:01.760436 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add19fb-e6a7-4e62-9176-7664e13dc53c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:01.793897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f56dcc-65b6-41d8-8592-f2f12f880ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:01.793897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f56dcc-65b6-41d8-8592-f2f12f880ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:01.794289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f56dcc-65b6-41d8-8592-f2f12f880ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:01.794463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f56dcc-65b6-41d8-8592-f2f12f880ef4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:01.805025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f56dcc-65b6-41d8-8592-f2f12f880ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-68f56dcc-65b6-41d8-8592-f2f12f880ef4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:25:01.813577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:01.814625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:01.815639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:01.815639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:01.815639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:01.815639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:01.815639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:25:01.815639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:25:01.815639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:25:01.815639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:25:01.815639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:25:01.815639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:25:01.815639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:25:01.815639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:01.815639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 603/1208] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:25:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:01.821235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13e029-da06-42c0-a179-5fb3594257af None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:01.860363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:01.917373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13e029-da06-42c0-a179-5fb3594257af None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:01.952136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033205cb-a4f7-4389-af6f-8a31ebdef777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:01.952801 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033205cb-a4f7-4389-af6f-8a31ebdef777 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:01.953110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033205cb-a4f7-4389-af6f-8a31ebdef777 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:01.953394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033205cb-a4f7-4389-af6f-8a31ebdef777 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:01.954140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033205cb-a4f7-4389-af6f-8a31ebdef777 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:01.954449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033205cb-a4f7-4389-af6f-8a31ebdef777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:01.955823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033205cb-a4f7-4389-af6f-8a31ebdef777 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:01.963931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 606/1209] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:25:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:01.969630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f56dcc-65b6-41d8-8592-f2f12f880ef4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:02.018205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:02.067656 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f56dcc-65b6-41d8-8592-f2f12f880ef4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:02.100640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8f7be-55e3-4b73-8a0d-5f8abd11df25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:02.101448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8f7be-55e3-4b73-8a0d-5f8abd11df25 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:02.101838 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8f7be-55e3-4b73-8a0d-5f8abd11df25 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:02.102015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8f7be-55e3-4b73-8a0d-5f8abd11df25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:02.112516 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8f7be-55e3-4b73-8a0d-5f8abd11df25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:02.113975 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8f7be-55e3-4b73-8a0d-5f8abd11df25 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:02.132182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-1ad8f7be-55e3-4b73-8a0d-5f8abd11df25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:02.132182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:02.132182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:02.132182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:02.132182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:02.132182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:02.132182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:02.132182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:02.132182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:02.132182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:02.132182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:02.132182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:02.132182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:25:02.132182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:25:02.132182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:02.136106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:02.137321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:25:02.137321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:25:02.137321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:02.137321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:02.137321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 604/1210] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:02.139928 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033205cb-a4f7-4389-af6f-8a31ebdef777 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:02.236450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:02.239297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033205cb-a4f7-4389-af6f-8a31ebdef777 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:02.272462 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b21bfe-fc63-4e98-aec8-2766a212d867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:02.273146 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b21bfe-fc63-4e98-aec8-2766a212d867 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:02.273466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b21bfe-fc63-4e98-aec8-2766a212d867 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:02.273718 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b21bfe-fc63-4e98-aec8-2766a212d867 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:02.274432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b21bfe-fc63-4e98-aec8-2766a212d867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:02.274741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b21bfe-fc63-4e98-aec8-2766a212d867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:02.276110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b21bfe-fc63-4e98-aec8-2766a212d867 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:02.287050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 607/1211] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:02.293522 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8f7be-55e3-4b73-8a0d-5f8abd11df25 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:02.392333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8f7be-55e3-4b73-8a0d-5f8abd11df25 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:02.393808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:02.429229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b889b91-0105-4bc7-8eb8-051a550fc10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:02.430040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b889b91-0105-4bc7-8eb8-051a550fc10f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:02.430329 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b889b91-0105-4bc7-8eb8-051a550fc10f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:02.430609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b889b91-0105-4bc7-8eb8-051a550fc10f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:02.437976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b889b91-0105-4bc7-8eb8-051a550fc10f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:02.439407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b889b91-0105-4bc7-8eb8-051a550fc10f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-7b889b91-0105-4bc7-8eb8-051a550fc10f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:25:02.454458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:02.455659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:02.456935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:02.456935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:02.456935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:25:02.456935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:25:02.456935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:25:02.456935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:25:02.456935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:25:02.456935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:25:02.456935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:25:02.456935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:02.458214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 605/1212] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:02.461726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b21bfe-fc63-4e98-aec8-2766a212d867 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:02.558989 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b21bfe-fc63-4e98-aec8-2766a212d867 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:02.591404 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525401c-d1b4-4ab9-8bba-b95d564456be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:02.592075 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525401c-d1b4-4ab9-8bba-b95d564456be None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:02.592362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525401c-d1b4-4ab9-8bba-b95d564456be None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:02.592644 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525401c-d1b4-4ab9-8bba-b95d564456be None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:02.593508 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525401c-d1b4-4ab9-8bba-b95d564456be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:02.593819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525401c-d1b4-4ab9-8bba-b95d564456be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:02.595258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525401c-d1b4-4ab9-8bba-b95d564456be None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:02.612512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 608/1213] 10.222.0.64 () {56 vars in 1143 bytes} [Mon May 23 15:25:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:02.618305 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b889b91-0105-4bc7-8eb8-051a550fc10f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:02.717430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b889b91-0105-4bc7-8eb8-051a550fc10f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:02.751170 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d95c1-b85d-4484-af19-753361bca3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:02.751824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d95c1-b85d-4484-af19-753361bca3da None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:02.752470 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d95c1-b85d-4484-af19-753361bca3da None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:02.752470 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d95c1-b85d-4484-af19-753361bca3da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:02.782325 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:02.818434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d95c1-b85d-4484-af19-753361bca3da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:02.819874 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d95c1-b85d-4484-af19-753361bca3da None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:02.918460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 606/1214] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:25:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:25:02.924718 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525401c-d1b4-4ab9-8bba-b95d564456be None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:02.943588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:03.024848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525401c-d1b4-4ab9-8bba-b95d564456be None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:03.059343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f2039-521a-416f-8d43-c3d075d87f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:03.060140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f2039-521a-416f-8d43-c3d075d87f0c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:03.060431 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f2039-521a-416f-8d43-c3d075d87f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:03.060714 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f2039-521a-416f-8d43-c3d075d87f0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:03.070727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f2039-521a-416f-8d43-c3d075d87f0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:03.072777 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f2039-521a-416f-8d43-c3d075d87f0c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:03.090889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-4d8f2039-521a-416f-8d43-c3d075d87f0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:03.090889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:03.090889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:03.090889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:03.090889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:03.090889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:03.090889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:03.090889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:03.090889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:03.090889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:03.090889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:03.090889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:03.090889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:25:03.090889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:25:03.090889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:03.091963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:03.093315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:25:03.093315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:25:03.093315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:03.093315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:25:03.093315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 609/1215] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:03.097835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d95c1-b85d-4484-af19-753361bca3da None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:03.158970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:03.197643 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d95c1-b85d-4484-af19-753361bca3da None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:03.229451 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6588f78-7055-4b2f-af5e-71b4e192f8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:03.230106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6588f78-7055-4b2f-af5e-71b4e192f8ed None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:03.230391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6588f78-7055-4b2f-af5e-71b4e192f8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:03.230674 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6588f78-7055-4b2f-af5e-71b4e192f8ed None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:03.231384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6588f78-7055-4b2f-af5e-71b4e192f8ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:03.231690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6588f78-7055-4b2f-af5e-71b4e192f8ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:03.233041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6588f78-7055-4b2f-af5e-71b4e192f8ed None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:03.244618 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 607/1216] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:03.250249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f2039-521a-416f-8d43-c3d075d87f0c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:03.320518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:03.350334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f2039-521a-416f-8d43-c3d075d87f0c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:03.382988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4a148-93bf-44ad-bea9-60fd741dde3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:03.383770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4a148-93bf-44ad-bea9-60fd741dde3d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:03.384052 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4a148-93bf-44ad-bea9-60fd741dde3d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:03.384327 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4a148-93bf-44ad-bea9-60fd741dde3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:03.392064 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4a148-93bf-44ad-bea9-60fd741dde3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:03.393567 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4a148-93bf-44ad-bea9-60fd741dde3d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:03.410197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-1fa4a148-93bf-44ad-bea9-60fd741dde3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:25:03.410197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:03.410197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:03.410197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:03.410197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:03.410197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:03.410197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:03.410197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:03.410197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:03.410197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:03.410197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:03.410197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:03.410197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:25:03.410197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:25:03.410197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:03.411265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:03.412441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:25:03.412441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:25:03.412441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:25:03.412441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:25:03.412441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:25:03.412441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:25:03.412441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:25:03.412441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:25:03.412441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 610/1217] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:25:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:03.417716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6588f78-7055-4b2f-af5e-71b4e192f8ed None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:03.518571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6588f78-7055-4b2f-af5e-71b4e192f8ed None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:03.540521 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:03.552231 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6cc25-0701-4acd-a717-5d6087a331ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:03.552893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6cc25-0701-4acd-a717-5d6087a331ea None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:03.553211 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6cc25-0701-4acd-a717-5d6087a331ea None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:03.553493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6cc25-0701-4acd-a717-5d6087a331ea None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:03.554281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6cc25-0701-4acd-a717-5d6087a331ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:03.554594 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6cc25-0701-4acd-a717-5d6087a331ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:03.556025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6cc25-0701-4acd-a717-5d6087a331ea None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:03.567707 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 608/1218] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:25:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:03.574719 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4a148-93bf-44ad-bea9-60fd741dde3d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:03.672242 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4a148-93bf-44ad-bea9-60fd741dde3d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:03.700886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:03.705811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb40fe6-e4f5-490a-8e51-35093a09605b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:03.706467 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb40fe6-e4f5-490a-8e51-35093a09605b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:03.706816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb40fe6-e4f5-490a-8e51-35093a09605b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:03.707092 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb40fe6-e4f5-490a-8e51-35093a09605b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:03.707917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb40fe6-e4f5-490a-8e51-35093a09605b None admin] RBAC: Adding query filter params (role.id=3d6e986dd2264e3cb7e01e25dbac6f97, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=04fbaeb7583a456d81cf79f6b101fb1b) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:03.708229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb40fe6-e4f5-490a-8e51-35093a09605b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:03.709733 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb40fe6-e4f5-490a-8e51-35093a09605b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:03.715969 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 611/1219] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:25:03 2022] GET /identity/v3/role_assignments?user.id=04fbaeb7583a456d81cf79f6b101fb1b&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=3d6e986dd2264e3cb7e01e25dbac6f97 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:05.886612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac0f71-f65b-4c16-871c-1c0a4d95165c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:05.887164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac0f71-f65b-4c16-871c-1c0a4d95165c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:05.887164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac0f71-f65b-4c16-871c-1c0a4d95165c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:05.888560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 609/1220] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:25:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:25:05.893928 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8a4dc-b604-469e-bacf-451495b6410a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:05.896851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8a4dc-b604-469e-bacf-451495b6410a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:05.896851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8a4dc-b604-469e-bacf-451495b6410a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:05.911650 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fce8a4dc-b604-469e-bacf-451495b6410a None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:25:05.912826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fce8a4dc-b604-469e-bacf-451495b6410a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:05.980185 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 612/1221] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:25:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:25:06.002247 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac0f71-f65b-4c16-871c-1c0a4d95165c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:06.101616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac0f71-f65b-4c16-871c-1c0a4d95165c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:06.134476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929a9ad-1290-4f66-8480-e08b2707da55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:06.135959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929a9ad-1290-4f66-8480-e08b2707da55 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:06.136239 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929a9ad-1290-4f66-8480-e08b2707da55 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:06.136518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929a9ad-1290-4f66-8480-e08b2707da55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:06.146602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929a9ad-1290-4f66-8480-e08b2707da55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-4929a9ad-1290-4f66-8480-e08b2707da55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:25:06.154450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:06.155478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:06.156408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:06.156408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:06.156408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:06.156408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:06.156408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:25:06.156408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:25:06.156408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:25:06.156408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:25:06.156408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:25:06.156408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:25:06.156408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:25:06.156408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:06.156408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 610/1222] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:25:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:06.162032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8a4dc-b604-469e-bacf-451495b6410a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:06.261430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8a4dc-b604-469e-bacf-451495b6410a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:06.284072 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:06.294429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334a692-3efa-49ca-af9c-2f0ef22adb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:06.295105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334a692-3efa-49ca-af9c-2f0ef22adb64 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:06.295392 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334a692-3efa-49ca-af9c-2f0ef22adb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:06.295675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334a692-3efa-49ca-af9c-2f0ef22adb64 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:06.296418 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334a692-3efa-49ca-af9c-2f0ef22adb64 None admin] RBAC: Adding query filter params (name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:06.296731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334a692-3efa-49ca-af9c-2f0ef22adb64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:06.298155 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334a692-3efa-49ca-af9c-2f0ef22adb64 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:06.306784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 613/1223] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:25:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:06.312628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929a9ad-1290-4f66-8480-e08b2707da55 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:06.356504 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:06.411857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929a9ad-1290-4f66-8480-e08b2707da55 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:06.445201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69df93f-44ab-4885-8d0b-8640befe7e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:06.445993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69df93f-44ab-4885-8d0b-8640befe7e64 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:06.446282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69df93f-44ab-4885-8d0b-8640befe7e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:06.446561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69df93f-44ab-4885-8d0b-8640befe7e64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:06.457742 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69df93f-44ab-4885-8d0b-8640befe7e64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:06.459184 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69df93f-44ab-4885-8d0b-8640befe7e64 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:06.476556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d69df93f-44ab-4885-8d0b-8640befe7e64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:06.476556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:06.476556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:06.476556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:06.476556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:06.476556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:06.476556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:06.476556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:06.476556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:06.476556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:06.476556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:06.476556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:06.476556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:25:06.476556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:25:06.476556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:06.477588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:06.479227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:25:06.479227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:25:06.479227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:06.479227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:06.479227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 611/1224] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:06.484246 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334a692-3efa-49ca-af9c-2f0ef22adb64 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:06.583999 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334a692-3efa-49ca-af9c-2f0ef22adb64 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:06.617387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c6256-870d-40fc-a0b6-7961c88e3fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:06.618097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c6256-870d-40fc-a0b6-7961c88e3fec None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:06.618387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c6256-870d-40fc-a0b6-7961c88e3fec None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:06.618669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c6256-870d-40fc-a0b6-7961c88e3fec None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:06.619887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c6256-870d-40fc-a0b6-7961c88e3fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:06.620201 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c6256-870d-40fc-a0b6-7961c88e3fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:06.621624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c6256-870d-40fc-a0b6-7961c88e3fec None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:06.633123 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 614/1225] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:06.639416 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69df93f-44ab-4885-8d0b-8640befe7e64 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:06.734330 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:06.740889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69df93f-44ab-4885-8d0b-8640befe7e64 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:06.774974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e0ce9-9c1d-4f5f-8f94-86fe6d978bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:06.776319 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e0ce9-9c1d-4f5f-8f94-86fe6d978bfa None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:06.776607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e0ce9-9c1d-4f5f-8f94-86fe6d978bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:06.776886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e0ce9-9c1d-4f5f-8f94-86fe6d978bfa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:06.784536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e0ce9-9c1d-4f5f-8f94-86fe6d978bfa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:06.786011 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e0ce9-9c1d-4f5f-8f94-86fe6d978bfa None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d62e0ce9-9c1d-4f5f-8f94-86fe6d978bfa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:25:06.800617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:06.801880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:06.803105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:06.803105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:06.803105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:25:06.803105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:25:06.803105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:25:06.803105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:25:06.803105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:25:06.803105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:25:06.803105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:25:06.803105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:06.805211 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 612/1226] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:06.807644 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c6256-870d-40fc-a0b6-7961c88e3fec None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:06.854071 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:06.905742 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c6256-870d-40fc-a0b6-7961c88e3fec None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:06.939424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9ca38-c5da-4fc7-b2fc-6c021d93359d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:06.940096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9ca38-c5da-4fc7-b2fc-6c021d93359d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:06.940385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9ca38-c5da-4fc7-b2fc-6c021d93359d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:06.940667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9ca38-c5da-4fc7-b2fc-6c021d93359d None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:06.941535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9ca38-c5da-4fc7-b2fc-6c021d93359d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:06.941853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9ca38-c5da-4fc7-b2fc-6c021d93359d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:06.943378 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9ca38-c5da-4fc7-b2fc-6c021d93359d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:06.960231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 615/1227] 10.222.0.64 () {56 vars in 1143 bytes} [Mon May 23 15:25:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:06.966596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e0ce9-9c1d-4f5f-8f94-86fe6d978bfa None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:07.068307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e0ce9-9c1d-4f5f-8f94-86fe6d978bfa None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:07.100395 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abdaa9-aa25-4a1d-b332-5b17dd3e6295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:07.101212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abdaa9-aa25-4a1d-b332-5b17dd3e6295 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:07.101499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abdaa9-aa25-4a1d-b332-5b17dd3e6295 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:07.101777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abdaa9-aa25-4a1d-b332-5b17dd3e6295 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:07.112451 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abdaa9-aa25-4a1d-b332-5b17dd3e6295 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:07.113843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abdaa9-aa25-4a1d-b332-5b17dd3e6295 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:07.130446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d9abdaa9-aa25-4a1d-b332-5b17dd3e6295 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:07.130446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:07.130446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:07.130446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:07.130446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:07.130446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:07.130446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:07.130446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:07.130446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:07.130446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:07.130446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:07.130446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:07.130446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:25:07.130446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:25:07.130446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:07.131563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:07.132731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:25:07.132731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:25:07.132731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:07.132731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:07.132731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 613/1228] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:07.137551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9ca38-c5da-4fc7-b2fc-6c021d93359d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:07.242464 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9ca38-c5da-4fc7-b2fc-6c021d93359d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:07.244786 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:07.277402 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff3a05-db32-4818-9610-1373d0403e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:07.278095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff3a05-db32-4818-9610-1373d0403e0c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:07.278380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff3a05-db32-4818-9610-1373d0403e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:07.278657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff3a05-db32-4818-9610-1373d0403e0c None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:07.279369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff3a05-db32-4818-9610-1373d0403e0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:07.279677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff3a05-db32-4818-9610-1373d0403e0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:07.281040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff3a05-db32-4818-9610-1373d0403e0c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:07.292052 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 616/1229] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:07.297822 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abdaa9-aa25-4a1d-b332-5b17dd3e6295 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:07.307634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:07.398111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abdaa9-aa25-4a1d-b332-5b17dd3e6295 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:07.431755 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538734c-d4bc-447d-93ad-3530c580bd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:07.432551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538734c-d4bc-447d-93ad-3530c580bd15 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:07.432835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538734c-d4bc-447d-93ad-3530c580bd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:07.433139 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538734c-d4bc-447d-93ad-3530c580bd15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:07.439579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538734c-d4bc-447d-93ad-3530c580bd15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:07.441002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538734c-d4bc-447d-93ad-3530c580bd15 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:07.455422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-7538734c-d4bc-447d-93ad-3530c580bd15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:25:07.455422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:07.455422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:07.455422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:07.455422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:07.455422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:07.455422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:07.455422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:07.455422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:07.455422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:07.455422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:07.455422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:07.455422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:25:07.455422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:25:07.455422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:07.464384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:07.465580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:25:07.465580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:25:07.465580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:25:07.465580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:25:07.465580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:25:07.465580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:25:07.465580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:25:07.465580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:07.465580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 614/1230] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:25:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:07.466216 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff3a05-db32-4818-9610-1373d0403e0c None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:07.562873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff3a05-db32-4818-9610-1373d0403e0c None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:07.595939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d31f6e-76bf-4d69-a892-c77b4425d673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:07.596602 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d31f6e-76bf-4d69-a892-c77b4425d673 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:07.596894 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d31f6e-76bf-4d69-a892-c77b4425d673 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:07.597211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d31f6e-76bf-4d69-a892-c77b4425d673 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:07.597999 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d31f6e-76bf-4d69-a892-c77b4425d673 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:07.598314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d31f6e-76bf-4d69-a892-c77b4425d673 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:07.599756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d31f6e-76bf-4d69-a892-c77b4425d673 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:07.611806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 617/1231] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:25:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:07.617722 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538734c-d4bc-447d-93ad-3530c580bd15 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:07.682410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:07.716146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538734c-d4bc-447d-93ad-3530c580bd15 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:07.749023 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76117a9-f20b-479f-9729-cbb65796d763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:07.749930 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76117a9-f20b-479f-9729-cbb65796d763 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:07.750214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76117a9-f20b-479f-9729-cbb65796d763 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:07.750490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76117a9-f20b-479f-9729-cbb65796d763 None admin] PATH_INFO: `/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/04fbaeb7583a456d81cf79f6b101fb1b/roles/3d6e986dd2264e3cb7e01e25dbac6f97` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:07.764444 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76117a9-f20b-479f-9729-cbb65796d763 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d81eafac474f37a71ee5cc292d172f, user_id=04fbaeb7583a456d81cf79f6b101fb1b, role_id=3d6e986dd2264e3cb7e01e25dbac6f97)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:07.765904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76117a9-f20b-479f-9729-cbb65796d763 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:07.938814 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 615/1232] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:25:07 2022] PUT /identity/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/04fbaeb7583a456d81cf79f6b101fb1b/roles/3d6e986dd2264e3cb7e01e25dbac6f97 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:25:10.123763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b52e34-1b4e-4284-94a9-7e2351a69fa7 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:10.124321 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b52e34-1b4e-4284-94a9-7e2351a69fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:10.124321 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b52e34-1b4e-4284-94a9-7e2351a69fa7 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:10.125766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 618/1233] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:25:10.130886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386bad4-f30d-4766-a6bc-2cc1ed17e1cf None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:10.131310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386bad4-f30d-4766-a6bc-2cc1ed17e1cf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:10.131420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386bad4-f30d-4766-a6bc-2cc1ed17e1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:10.147871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2386bad4-f30d-4766-a6bc-2cc1ed17e1cf None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:25:10.149181 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2386bad4-f30d-4766-a6bc-2cc1ed17e1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:10.214016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 616/1234] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:25:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:25:10.239147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b52e34-1b4e-4284-94a9-7e2351a69fa7 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:10.269907 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:10.346857 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b52e34-1b4e-4284-94a9-7e2351a69fa7 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:10.381543 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69082d26-6894-4b4c-9bd0-48c96600ed5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:10.382271 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69082d26-6894-4b4c-9bd0-48c96600ed5a None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:10.382556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69082d26-6894-4b4c-9bd0-48c96600ed5a None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:10.382837 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69082d26-6894-4b4c-9bd0-48c96600ed5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:10.392774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69082d26-6894-4b4c-9bd0-48c96600ed5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-69082d26-6894-4b4c-9bd0-48c96600ed5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:25:10.400502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:10.402305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:10.403941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:10.403941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:10.403941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:10.403941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:10.403941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:25:10.403941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:25:10.403941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:25:10.403941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:25:10.403941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:25:10.403941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:25:10.403941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:25:10.403941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:25:10.403941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 619/1235] 10.222.0.64 () {56 vars in 1102 bytes} [Mon May 23 15:25:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:10.407749 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386bad4-f30d-4766-a6bc-2cc1ed17e1cf None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:10.453291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:10.504399 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386bad4-f30d-4766-a6bc-2cc1ed17e1cf None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:10.535891 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e947242-8154-46fe-802b-6fa97bb34b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:10.536549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e947242-8154-46fe-802b-6fa97bb34b04 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:10.536877 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e947242-8154-46fe-802b-6fa97bb34b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:10.537186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e947242-8154-46fe-802b-6fa97bb34b04 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:10.537929 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e947242-8154-46fe-802b-6fa97bb34b04 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:10.538240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e947242-8154-46fe-802b-6fa97bb34b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:10.539596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e947242-8154-46fe-802b-6fa97bb34b04 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:10.548429 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 617/1236] 10.222.0.64 () {56 vars in 1103 bytes} [Mon May 23 15:25:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:10.554708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69082d26-6894-4b4c-9bd0-48c96600ed5a None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:10.636789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:10.652590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69082d26-6894-4b4c-9bd0-48c96600ed5a None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:10.685175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03a4cc-4e6e-4255-b8fa-c1875f2a1d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:10.686074 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03a4cc-4e6e-4255-b8fa-c1875f2a1d3b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:10.686358 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03a4cc-4e6e-4255-b8fa-c1875f2a1d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:10.686633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03a4cc-4e6e-4255-b8fa-c1875f2a1d3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:10.696659 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03a4cc-4e6e-4255-b8fa-c1875f2a1d3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:10.698828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03a4cc-4e6e-4255-b8fa-c1875f2a1d3b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:10.716385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-6e03a4cc-4e6e-4255-b8fa-c1875f2a1d3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:10.716385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:10.716385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:10.716385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:10.716385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:10.716385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:10.716385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:10.716385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:10.716385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:10.716385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:10.716385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:10.716385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:10.716385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:25:10.716385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:25:10.716385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:10.717566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:10.723154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:25:10.723154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:25:10.723154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:10.723154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:25:10.723154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 620/1237] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:10.723452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e947242-8154-46fe-802b-6fa97bb34b04 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:10.819386 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e947242-8154-46fe-802b-6fa97bb34b04 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:10.851916 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be21c9-9ba6-403d-9625-1a24378552e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:10.852612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be21c9-9ba6-403d-9625-1a24378552e5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:10.852897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be21c9-9ba6-403d-9625-1a24378552e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:10.853234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be21c9-9ba6-403d-9625-1a24378552e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:10.854061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be21c9-9ba6-403d-9625-1a24378552e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:10.854367 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be21c9-9ba6-403d-9625-1a24378552e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:10.855718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be21c9-9ba6-403d-9625-1a24378552e5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:10.866648 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 618/1238] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:10.872903 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03a4cc-4e6e-4255-b8fa-c1875f2a1d3b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:10.970059 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03a4cc-4e6e-4255-b8fa-c1875f2a1d3b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:11.005923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2e98f-5e77-46af-997d-ab1c82be3c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:11.006727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2e98f-5e77-46af-997d-ab1c82be3c75 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:11.007019 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2e98f-5e77-46af-997d-ab1c82be3c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:11.007305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2e98f-5e77-46af-997d-ab1c82be3c75 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:11.009993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:11.014447 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2e98f-5e77-46af-997d-ab1c82be3c75 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:11.015890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2e98f-5e77-46af-997d-ab1c82be3c75 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-71a2e98f-5e77-46af-997d-ab1c82be3c75 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:25:11.030493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:11.031669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:11.032696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:11.032696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:11.032696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:25:11.032696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:25:11.032696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:25:11.032696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:25:11.032696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:25:11.032696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:25:11.032696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:25:11.032696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:25:11.032696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 621/1239] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:11.037685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be21c9-9ba6-403d-9625-1a24378552e5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:11.049476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:11.140332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be21c9-9ba6-403d-9625-1a24378552e5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:11.172775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f4a4e-3714-44eb-84cc-9e4e09bcf480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:11.173496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f4a4e-3714-44eb-84cc-9e4e09bcf480 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:11.173786 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f4a4e-3714-44eb-84cc-9e4e09bcf480 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:11.174328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f4a4e-3714-44eb-84cc-9e4e09bcf480 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:11.175158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f4a4e-3714-44eb-84cc-9e4e09bcf480 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:11.175534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f4a4e-3714-44eb-84cc-9e4e09bcf480 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:11.176958 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f4a4e-3714-44eb-84cc-9e4e09bcf480 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:11.194222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 619/1240] 10.222.0.64 () {56 vars in 1143 bytes} [Mon May 23 15:25:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:11.200053 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2e98f-5e77-46af-997d-ab1c82be3c75 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:11.304472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2e98f-5e77-46af-997d-ab1c82be3c75 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:11.338984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0bab36-ccee-45e9-854b-714c3dc8d488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:11.339630 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0bab36-ccee-45e9-854b-714c3dc8d488 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:11.339914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0bab36-ccee-45e9-854b-714c3dc8d488 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:11.340191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0bab36-ccee-45e9-854b-714c3dc8d488 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:11.398948 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:11.405947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0bab36-ccee-45e9-854b-714c3dc8d488 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:11.407498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0bab36-ccee-45e9-854b-714c3dc8d488 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:11.504442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 622/1241] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:25:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:25:11.510535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f4a4e-3714-44eb-84cc-9e4e09bcf480 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:11.537285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:11.606274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f4a4e-3714-44eb-84cc-9e4e09bcf480 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:11.642940 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70530ad3-b3b2-4a55-8656-aa87b3f5c745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:11.643739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70530ad3-b3b2-4a55-8656-aa87b3f5c745 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:11.644026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70530ad3-b3b2-4a55-8656-aa87b3f5c745 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:11.644302 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70530ad3-b3b2-4a55-8656-aa87b3f5c745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:11.655048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70530ad3-b3b2-4a55-8656-aa87b3f5c745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:11.656469 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70530ad3-b3b2-4a55-8656-aa87b3f5c745 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:11.673960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-70530ad3-b3b2-4a55-8656-aa87b3f5c745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:11.673960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:11.673960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:11.673960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:11.673960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:11.673960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:11.673960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:11.673960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:11.673960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:11.673960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:11.673960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:11.673960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:11.673960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:25:11.673960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:25:11.673960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:11.675301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:11.686626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:25:11.686626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:25:11.686626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:11.686626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:11.686626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 620/1242] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:25:11.687012 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0bab36-ccee-45e9-854b-714c3dc8d488 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:11.776530 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:11.782145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0bab36-ccee-45e9-854b-714c3dc8d488 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:11.814968 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491727c1-6a63-4bc9-bddf-cbdd5e010302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:11.815635 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491727c1-6a63-4bc9-bddf-cbdd5e010302 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:11.815925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491727c1-6a63-4bc9-bddf-cbdd5e010302 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:11.816208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491727c1-6a63-4bc9-bddf-cbdd5e010302 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:11.816931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491727c1-6a63-4bc9-bddf-cbdd5e010302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:11.817274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491727c1-6a63-4bc9-bddf-cbdd5e010302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:11.818703 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491727c1-6a63-4bc9-bddf-cbdd5e010302 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:11.830862 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 623/1243] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:11.836110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70530ad3-b3b2-4a55-8656-aa87b3f5c745 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:11.912220 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:11.932716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70530ad3-b3b2-4a55-8656-aa87b3f5c745 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:11.966998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7019e-857b-4082-a89b-5a45f4176134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:11.967802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7019e-857b-4082-a89b-5a45f4176134 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:11.968083 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7019e-857b-4082-a89b-5a45f4176134 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:11.968357 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7019e-857b-4082-a89b-5a45f4176134 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:11.975019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7019e-857b-4082-a89b-5a45f4176134 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:11.976437 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7019e-857b-4082-a89b-5a45f4176134 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:11.991405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-fca7019e-857b-4082-a89b-5a45f4176134 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:25:11.991405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:11.991405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:11.991405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:11.991405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:11.991405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:11.991405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:11.991405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:11.991405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:11.991405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:11.991405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:11.991405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:11.991405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:25:11.991405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:25:11.991405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:11.992530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:11.993751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:25:11.993751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:25:11.993751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:25:11.993751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:25:11.993751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:25:11.993751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:25:11.993751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:25:11.993751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:11.997219 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 621/1244] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:25:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:11.998840 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491727c1-6a63-4bc9-bddf-cbdd5e010302 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:12.097042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491727c1-6a63-4bc9-bddf-cbdd5e010302 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:12.129296 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a436f-0b94-4f4f-a740-7d1e97012bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:12.129955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a436f-0b94-4f4f-a740-7d1e97012bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:12.130246 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a436f-0b94-4f4f-a740-7d1e97012bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:12.130524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a436f-0b94-4f4f-a740-7d1e97012bf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:12.131302 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a436f-0b94-4f4f-a740-7d1e97012bf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:12.131613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a436f-0b94-4f4f-a740-7d1e97012bf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:12.133045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a436f-0b94-4f4f-a740-7d1e97012bf7 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:12.145172 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 624/1245] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:25:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:12.151583 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7019e-857b-4082-a89b-5a45f4176134 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:12.251773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7019e-857b-4082-a89b-5a45f4176134 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:12.285626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08038d03-586b-47ea-942f-9d40c70eef92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:12.286291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08038d03-586b-47ea-942f-9d40c70eef92 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:12.286628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08038d03-586b-47ea-942f-9d40c70eef92 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:12.286924 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08038d03-586b-47ea-942f-9d40c70eef92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:12.287758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08038d03-586b-47ea-942f-9d40c70eef92 None admin] RBAC: Adding query filter params (role.id=3d6e986dd2264e3cb7e01e25dbac6f97, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=04fbaeb7583a456d81cf79f6b101fb1b) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:12.288076 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08038d03-586b-47ea-942f-9d40c70eef92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:12.289559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08038d03-586b-47ea-942f-9d40c70eef92 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:12.295599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 622/1246] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:25:12 2022] GET /identity/v3/role_assignments?user.id=04fbaeb7583a456d81cf79f6b101fb1b&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=3d6e986dd2264e3cb7e01e25dbac6f97 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:14.540169 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7396f4-dcec-4348-9f7b-be8d7edffbbe None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:14.540800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7396f4-dcec-4348-9f7b-be8d7edffbbe None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:14.540927 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7396f4-dcec-4348-9f7b-be8d7edffbbe None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:14.542236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 625/1247] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:25:14.547504 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e21fa9-e64a-4617-bb66-155e30ecf774 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:14.547924 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e21fa9-e64a-4617-bb66-155e30ecf774 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:14.548104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e21fa9-e64a-4617-bb66-155e30ecf774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:14.553428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:14.565972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b0e21fa9-e64a-4617-bb66-155e30ecf774 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:25:14.567170 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b0e21fa9-e64a-4617-bb66-155e30ecf774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:14.634241 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 623/1248] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:25:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:25:14.658279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7396f4-dcec-4348-9f7b-be8d7edffbbe None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:14.676459 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:14.761558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7396f4-dcec-4348-9f7b-be8d7edffbbe None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:14.794914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bd2aa-f45c-4765-8e5b-da96836f40ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:14.795646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bd2aa-f45c-4765-8e5b-da96836f40ff None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:14.795932 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bd2aa-f45c-4765-8e5b-da96836f40ff None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:14.796210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bd2aa-f45c-4765-8e5b-da96836f40ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:14.807099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bd2aa-f45c-4765-8e5b-da96836f40ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-b27bd2aa-f45c-4765-8e5b-da96836f40ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:25:14.814712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:14.817299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:14.818386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:14.818386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:14.818386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:14.818386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:14.818386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:25:14.818386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:25:14.818386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:25:14.818386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:25:14.818386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:25:14.818386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:25:14.818386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:25:14.818386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:25:14.818386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 626/1249] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:25:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:14.821958 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e21fa9-e64a-4617-bb66-155e30ecf774 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:14.918615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e21fa9-e64a-4617-bb66-155e30ecf774 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:14.947969 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:14.952690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de94ac-85f1-4e45-9051-7a9e19a8d52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:14.953516 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de94ac-85f1-4e45-9051-7a9e19a8d52e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:14.953853 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de94ac-85f1-4e45-9051-7a9e19a8d52e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:14.954136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de94ac-85f1-4e45-9051-7a9e19a8d52e None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:14.954878 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de94ac-85f1-4e45-9051-7a9e19a8d52e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:14.955200 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de94ac-85f1-4e45-9051-7a9e19a8d52e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:14.956603 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de94ac-85f1-4e45-9051-7a9e19a8d52e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:14.964655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 624/1250] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:25:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:14.970351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bd2aa-f45c-4765-8e5b-da96836f40ff None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:15.046478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:15.068589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bd2aa-f45c-4765-8e5b-da96836f40ff None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:15.101836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80e474-17d8-4332-9786-dd1944c4b4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:15.102758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80e474-17d8-4332-9786-dd1944c4b4ff None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:15.103047 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80e474-17d8-4332-9786-dd1944c4b4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:15.104196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80e474-17d8-4332-9786-dd1944c4b4ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:15.114480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80e474-17d8-4332-9786-dd1944c4b4ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:15.115946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80e474-17d8-4332-9786-dd1944c4b4ff None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:15.133254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-bf80e474-17d8-4332-9786-dd1944c4b4ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:15.133254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:15.133254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:15.133254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:15.133254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:15.133254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:15.133254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:15.133254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:15.133254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:15.133254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:15.133254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:15.133254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:15.133254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:25:15.133254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:25:15.133254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:15.134310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:15.135611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:25:15.135611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:25:15.135611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:15.135611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:25:15.135611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 627/1251] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:15.140372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de94ac-85f1-4e45-9051-7a9e19a8d52e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:15.242371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de94ac-85f1-4e45-9051-7a9e19a8d52e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:15.275751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cfedf-7e13-4c36-9a26-ee13842a2f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:15.276381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cfedf-7e13-4c36-9a26-ee13842a2f7c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:15.276663 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cfedf-7e13-4c36-9a26-ee13842a2f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:15.276957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cfedf-7e13-4c36-9a26-ee13842a2f7c None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:15.277685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cfedf-7e13-4c36-9a26-ee13842a2f7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:15.277992 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cfedf-7e13-4c36-9a26-ee13842a2f7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:15.279344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cfedf-7e13-4c36-9a26-ee13842a2f7c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:15.291577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 625/1252] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:15.297408 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80e474-17d8-4332-9786-dd1944c4b4ff None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:15.396510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80e474-17d8-4332-9786-dd1944c4b4ff None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:15.422960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:15.430855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af919b1c-b5f1-46ad-bc7d-177a6144d7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:15.431681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af919b1c-b5f1-46ad-bc7d-177a6144d7ef None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:15.431963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af919b1c-b5f1-46ad-bc7d-177a6144d7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:15.432242 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af919b1c-b5f1-46ad-bc7d-177a6144d7ef None admin] PATH_INFO: `/v3/users/placement` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:15.439757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af919b1c-b5f1-46ad-bc7d-177a6144d7ef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:15.441273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af919b1c-b5f1-46ad-bc7d-177a6144d7ef None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-af919b1c-b5f1-46ad-bc7d-177a6144d7ef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:25:15.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:15.456571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:15.457694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:15.457694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:15.457694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:25:15.457694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:25:15.457694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:25:15.457694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:25:15.457694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:25:15.457694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:25:15.457694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:25:15.457694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:25:15.457694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 628/1253] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:15.463037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cfedf-7e13-4c36-9a26-ee13842a2f7c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:15.519430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:15.563070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cfedf-7e13-4c36-9a26-ee13842a2f7c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:15.596275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6feea4-4dc1-497c-9105-7645ab647358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:15.596943 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6feea4-4dc1-497c-9105-7645ab647358 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:15.597259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6feea4-4dc1-497c-9105-7645ab647358 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:15.597536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6feea4-4dc1-497c-9105-7645ab647358 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:15.598364 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6feea4-4dc1-497c-9105-7645ab647358 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:15.598674 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6feea4-4dc1-497c-9105-7645ab647358 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:15.600094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6feea4-4dc1-497c-9105-7645ab647358 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:15.616383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 626/1254] 10.222.0.64 () {56 vars in 1143 bytes} [Mon May 23 15:25:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:15.622798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af919b1c-b5f1-46ad-bc7d-177a6144d7ef None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:15.724708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af919b1c-b5f1-46ad-bc7d-177a6144d7ef None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:15.761410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb229956-d922-45b6-9cce-ea97180b6c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:15.762071 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb229956-d922-45b6-9cce-ea97180b6c0f None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:15.762360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb229956-d922-45b6-9cce-ea97180b6c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:15.762642 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb229956-d922-45b6-9cce-ea97180b6c0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:15.817185 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:15.829908 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb229956-d922-45b6-9cce-ea97180b6c0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:15.831366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb229956-d922-45b6-9cce-ea97180b6c0f None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:15.929214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 629/1255] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:25:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:25:15.935542 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6feea4-4dc1-497c-9105-7645ab647358 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:16.030967 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6feea4-4dc1-497c-9105-7645ab647358 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:16.043743 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:16.063965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b9074-2272-4872-9852-2b04b67edf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:16.064751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b9074-2272-4872-9852-2b04b67edf45 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:16.065037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b9074-2272-4872-9852-2b04b67edf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:16.065341 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b9074-2272-4872-9852-2b04b67edf45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:16.075699 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b9074-2272-4872-9852-2b04b67edf45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:16.077097 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b9074-2272-4872-9852-2b04b67edf45 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:16.094858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-ec6b9074-2272-4872-9852-2b04b67edf45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:16.094858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:16.094858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:16.094858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:16.094858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:16.094858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:16.094858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:16.094858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:16.094858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:16.094858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:16.094858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:16.094858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:16.094858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:25:16.094858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:25:16.094858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:16.095933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:16.096945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:25:16.096945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:25:16.096945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:16.096945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:16.096945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 627/1256] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:16.101986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb229956-d922-45b6-9cce-ea97180b6c0f None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:16.188418 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:16.200097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb229956-d922-45b6-9cce-ea97180b6c0f None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:16.234175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325872e7-9efe-424e-9e51-8dfea6d69373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:16.234838 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325872e7-9efe-424e-9e51-8dfea6d69373 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:16.235127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325872e7-9efe-424e-9e51-8dfea6d69373 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:16.235407 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325872e7-9efe-424e-9e51-8dfea6d69373 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:16.236128 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325872e7-9efe-424e-9e51-8dfea6d69373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:16.236437 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325872e7-9efe-424e-9e51-8dfea6d69373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:16.237837 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325872e7-9efe-424e-9e51-8dfea6d69373 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:16.248745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 630/1257] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:16.254536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b9074-2272-4872-9852-2b04b67edf45 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:16.363021 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b9074-2272-4872-9852-2b04b67edf45 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:16.394141 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74123fc-9ffc-44da-988f-d3f4495519b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:16.394940 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74123fc-9ffc-44da-988f-d3f4495519b9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:16.395228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74123fc-9ffc-44da-988f-d3f4495519b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:16.395502 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74123fc-9ffc-44da-988f-d3f4495519b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:16.403100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74123fc-9ffc-44da-988f-d3f4495519b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:16.404546 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74123fc-9ffc-44da-988f-d3f4495519b9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:16.418310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a74123fc-9ffc-44da-988f-d3f4495519b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:25:16.418310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:16.418310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:16.418310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:16.418310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:16.418310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:16.418310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:16.418310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:16.418310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:16.418310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:16.418310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:16.418310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:16.418310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:25:16.418310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:25:16.418310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:16.419462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:16.420680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:25:16.420680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:25:16.420680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:25:16.420680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:25:16.420680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:25:16.420680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:25:16.420680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:25:16.420680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:16.420680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 628/1258] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:25:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:16.425803 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325872e7-9efe-424e-9e51-8dfea6d69373 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:16.524838 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325872e7-9efe-424e-9e51-8dfea6d69373 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:16.558245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1b73c-20aa-4239-ac03-9d8795dcd8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:16.558947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1b73c-20aa-4239-ac03-9d8795dcd8b1 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:16.559240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1b73c-20aa-4239-ac03-9d8795dcd8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:16.559512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1b73c-20aa-4239-ac03-9d8795dcd8b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:16.560390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d1b73c-20aa-4239-ac03-9d8795dcd8b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:16.560720 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d1b73c-20aa-4239-ac03-9d8795dcd8b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:16.562196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d1b73c-20aa-4239-ac03-9d8795dcd8b1 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:16.574592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 631/1259] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:25:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:16.580966 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74123fc-9ffc-44da-988f-d3f4495519b9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:16.607884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:16.683273 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74123fc-9ffc-44da-988f-d3f4495519b9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:16.717863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1e9c7-c14a-4868-ba58-ee14dcf741d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:16.718519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1e9c7-c14a-4868-ba58-ee14dcf741d4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:16.718806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1e9c7-c14a-4868-ba58-ee14dcf741d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:16.719085 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1e9c7-c14a-4868-ba58-ee14dcf741d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:16.719948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1e9c7-c14a-4868-ba58-ee14dcf741d4 None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=04fbaeb7583a456d81cf79f6b101fb1b) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:16.720262 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1e9c7-c14a-4868-ba58-ee14dcf741d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:16.721768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1e9c7-c14a-4868-ba58-ee14dcf741d4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:16.728105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 629/1260] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:25:16 2022] GET /identity/v3/role_assignments?user.id=04fbaeb7583a456d81cf79f6b101fb1b&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:18.880141 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0dab5-58c2-4f8b-aa6f-ddf3a2c12221 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:18.880692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0dab5-58c2-4f8b-aa6f-ddf3a2c12221 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:18.880692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0dab5-58c2-4f8b-aa6f-ddf3a2c12221 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:18.882123 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 632/1261] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:25:18.888189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07268482-e5f8-4422-bf56-8a086e78ae63 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:18.888601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07268482-e5f8-4422-bf56-8a086e78ae63 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:18.888744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07268482-e5f8-4422-bf56-8a086e78ae63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:18.905268 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-07268482-e5f8-4422-bf56-8a086e78ae63 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:25:18.906439 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-07268482-e5f8-4422-bf56-8a086e78ae63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:18.972190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 630/1262] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:25:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:25:18.994531 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0dab5-58c2-4f8b-aa6f-ddf3a2c12221 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:19.005309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:19.097323 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0dab5-58c2-4f8b-aa6f-ddf3a2c12221 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:19.131110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b67a17-f047-407e-9fc9-28c8e42680ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:19.131904 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b67a17-f047-407e-9fc9-28c8e42680ec None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:19.132210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b67a17-f047-407e-9fc9-28c8e42680ec None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:19.132476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b67a17-f047-407e-9fc9-28c8e42680ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:19.143635 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b67a17-f047-407e-9fc9-28c8e42680ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-d7b67a17-f047-407e-9fc9-28c8e42680ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:25:19.151841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:19.163121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:19.164221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:19.164221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:19.164221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:19.164221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:19.164221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:25:19.164221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:25:19.164221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:25:19.164221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:25:19.164221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:25:19.164221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:25:19.164221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:25:19.164221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:25:19.164221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 633/1263] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:25:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:19.165163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07268482-e5f8-4422-bf56-8a086e78ae63 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:19.177107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:19.253844 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07268482-e5f8-4422-bf56-8a086e78ae63 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:19.286544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb36cdf-ad63-4f27-938d-35568fb81924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:19.287306 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb36cdf-ad63-4f27-938d-35568fb81924 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:19.287593 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb36cdf-ad63-4f27-938d-35568fb81924 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:19.287869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb36cdf-ad63-4f27-938d-35568fb81924 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:19.288605 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb36cdf-ad63-4f27-938d-35568fb81924 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:19.288911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb36cdf-ad63-4f27-938d-35568fb81924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:19.290297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb36cdf-ad63-4f27-938d-35568fb81924 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:19.301256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 631/1264] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:25:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:25:19.305204 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b67a17-f047-407e-9fc9-28c8e42680ec None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:19.372067 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:19.406328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b67a17-f047-407e-9fc9-28c8e42680ec None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:19.440967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543a018-087c-475f-91ce-292a07e8492e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:19.441788 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543a018-087c-475f-91ce-292a07e8492e None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:19.442077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543a018-087c-475f-91ce-292a07e8492e None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:19.442360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543a018-087c-475f-91ce-292a07e8492e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:19.453735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543a018-087c-475f-91ce-292a07e8492e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:19.455226 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543a018-087c-475f-91ce-292a07e8492e None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:19.472994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-8543a018-087c-475f-91ce-292a07e8492e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:19.472994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:19.472994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:19.472994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:19.472994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:19.472994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:19.472994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:19.472994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:19.472994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:19.472994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:19.472994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:19.472994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:19.472994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:25:19.472994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:25:19.472994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:19.474171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:19.475429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:25:19.475429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:25:19.475429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:19.475429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:25:19.475429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 634/1265] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:19.480799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb36cdf-ad63-4f27-938d-35568fb81924 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:19.561652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:19.580143 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb36cdf-ad63-4f27-938d-35568fb81924 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:19.612238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03ff93-c728-438b-bb79-ad92c784baf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:19.612899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03ff93-c728-438b-bb79-ad92c784baf3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:19.613226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03ff93-c728-438b-bb79-ad92c784baf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:19.613519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03ff93-c728-438b-bb79-ad92c784baf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:19.614238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03ff93-c728-438b-bb79-ad92c784baf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:19.614549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03ff93-c728-438b-bb79-ad92c784baf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:19.615915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03ff93-c728-438b-bb79-ad92c784baf3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:19.628138 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 632/1266] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:19.633819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543a018-087c-475f-91ce-292a07e8492e None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:19.739727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543a018-087c-475f-91ce-292a07e8492e None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:19.759001 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:19.772836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7c273-87ad-45fa-aacb-9f944cd23d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:19.773666 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7c273-87ad-45fa-aacb-9f944cd23d62 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:19.773953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7c273-87ad-45fa-aacb-9f944cd23d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:19.774228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7c273-87ad-45fa-aacb-9f944cd23d62 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:19.781521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7c273-87ad-45fa-aacb-9f944cd23d62 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:19.784391 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7c273-87ad-45fa-aacb-9f944cd23d62 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-69a7c273-87ad-45fa-aacb-9f944cd23d62 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:25:19.797020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:19.798194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:19.799861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:19.799861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:19.799861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:25:19.799861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:25:19.799861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:25:19.799861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:25:19.799861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:25:19.799861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:25:19.799861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:25:19.799861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:25:19.799861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 635/1267] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:19.804208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03ff93-c728-438b-bb79-ad92c784baf3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:19.903523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03ff93-c728-438b-bb79-ad92c784baf3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:19.936954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaeacf9-fe46-417e-a482-181b261953a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:19.937650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaeacf9-fe46-417e-a482-181b261953a8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:19.937984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaeacf9-fe46-417e-a482-181b261953a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:19.938266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaeacf9-fe46-417e-a482-181b261953a8 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:19.939091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaeacf9-fe46-417e-a482-181b261953a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:19.939403 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaeacf9-fe46-417e-a482-181b261953a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:19.940824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaeacf9-fe46-417e-a482-181b261953a8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:19.950048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:19.958156 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 633/1268] 10.222.0.64 () {56 vars in 1143 bytes} [Mon May 23 15:25:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:19.963855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7c273-87ad-45fa-aacb-9f944cd23d62 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:20.075608 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7c273-87ad-45fa-aacb-9f944cd23d62 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:20.108400 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ac7a3-3f67-44cf-aaae-0a9da35623d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:20.109219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ac7a3-3f67-44cf-aaae-0a9da35623d4 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:20.109508 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ac7a3-3f67-44cf-aaae-0a9da35623d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:20.109826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ac7a3-3f67-44cf-aaae-0a9da35623d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:20.120966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ac7a3-3f67-44cf-aaae-0a9da35623d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:20.122435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ac7a3-3f67-44cf-aaae-0a9da35623d4 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:20.139556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-407ac7a3-3f67-44cf-aaae-0a9da35623d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:20.139556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:20.139556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:20.139556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:20.139556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:20.139556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:20.139556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:20.139556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:20.139556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:20.139556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:20.139556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:20.139556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:20.139556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:25:20.139556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:25:20.139556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:20.140682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:20.141707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:25:20.141707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:25:20.141707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:20.141707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:25:20.141707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 636/1269] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:20.146561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaeacf9-fe46-417e-a482-181b261953a8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:20.243790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaeacf9-fe46-417e-a482-181b261953a8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:20.277306 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1779667-5fc4-4196-a02b-c047d428ee94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:20.277963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1779667-5fc4-4196-a02b-c047d428ee94 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:20.278248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1779667-5fc4-4196-a02b-c047d428ee94 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:20.278526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1779667-5fc4-4196-a02b-c047d428ee94 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:20.279240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1779667-5fc4-4196-a02b-c047d428ee94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:20.279546 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1779667-5fc4-4196-a02b-c047d428ee94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:20.280907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1779667-5fc4-4196-a02b-c047d428ee94 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:20.291955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 634/1270] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:20.297934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ac7a3-3f67-44cf-aaae-0a9da35623d4 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:20.334766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:20.404119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ac7a3-3f67-44cf-aaae-0a9da35623d4 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:20.439259 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63128a09-b582-43c5-93c6-804235cac38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:20.440048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63128a09-b582-43c5-93c6-804235cac38b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:20.440335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63128a09-b582-43c5-93c6-804235cac38b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:20.440652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63128a09-b582-43c5-93c6-804235cac38b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:20.448363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63128a09-b582-43c5-93c6-804235cac38b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:20.449848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63128a09-b582-43c5-93c6-804235cac38b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:20.464029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-63128a09-b582-43c5-93c6-804235cac38b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:25:20.464029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:20.464029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:20.464029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:20.464029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:20.464029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:20.464029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:20.464029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:20.464029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:20.464029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:20.464029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:20.464029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:20.464029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:25:20.464029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:25:20.464029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:20.470370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:20.471510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:25:20.471510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:25:20.471510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:25:20.471510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:25:20.471510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:25:20.471510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:25:20.471510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:25:20.471510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:25:20.471510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 637/1271] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:25:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:20.472114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1779667-5fc4-4196-a02b-c047d428ee94 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:20.531306 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:20.573351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1779667-5fc4-4196-a02b-c047d428ee94 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:20.605474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b50e0c-4458-4b6c-bda8-50d15e1a39c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:20.606128 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b50e0c-4458-4b6c-bda8-50d15e1a39c4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:20.606412 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b50e0c-4458-4b6c-bda8-50d15e1a39c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:20.606688 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b50e0c-4458-4b6c-bda8-50d15e1a39c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:20.607464 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b50e0c-4458-4b6c-bda8-50d15e1a39c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:20.607770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b50e0c-4458-4b6c-bda8-50d15e1a39c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:20.609211 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b50e0c-4458-4b6c-bda8-50d15e1a39c4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:20.621283 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 635/1272] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:25:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:20.627366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63128a09-b582-43c5-93c6-804235cac38b None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:20.722767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:20.732338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63128a09-b582-43c5-93c6-804235cac38b None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:20.766359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcdb7a4-9786-42c8-b8b5-ca8b6349b69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:20.767549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcdb7a4-9786-42c8-b8b5-ca8b6349b69d None admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:20.767549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcdb7a4-9786-42c8-b8b5-ca8b6349b69d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:20.767843 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcdb7a4-9786-42c8-b8b5-ca8b6349b69d None admin] PATH_INFO: `/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/04fbaeb7583a456d81cf79f6b101fb1b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:20.769777 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcdb7a4-9786-42c8-b8b5-ca8b6349b69d None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d81eafac474f37a71ee5cc292d172f, user_id=04fbaeb7583a456d81cf79f6b101fb1b, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:20.771209 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcdb7a4-9786-42c8-b8b5-ca8b6349b69d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:20.877978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 638/1273] 10.222.0.64 () {58 vars in 1439 bytes} [Mon May 23 15:25:20 2022] PUT /identity/v3/projects/d8d81eafac474f37a71ee5cc292d172f/users/04fbaeb7583a456d81cf79f6b101fb1b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:25:23.055990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4da02-ded5-4cb4-b139-fb7c159699fb None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:23.056655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4da02-ded5-4cb4-b139-fb7c159699fb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:23.056655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4da02-ded5-4cb4-b139-fb7c159699fb None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:23.057976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 636/1274] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:25:23.063345 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb369f-c8dc-407e-b6cf-83496bf5bcc6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:23.063785 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb369f-c8dc-407e-b6cf-83496bf5bcc6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:23.063948 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb369f-c8dc-407e-b6cf-83496bf5bcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:23.081377 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-cadb369f-c8dc-407e-b6cf-83496bf5bcc6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:25:23.085187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cadb369f-c8dc-407e-b6cf-83496bf5bcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:23.149086 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 639/1275] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:25:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:25:23.173556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4da02-ded5-4cb4-b139-fb7c159699fb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:23.273890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4da02-ded5-4cb4-b139-fb7c159699fb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:23.291455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:23.308609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958ad2f-fc73-4bc8-9e9a-9166bf0aca28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:23.309416 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958ad2f-fc73-4bc8-9e9a-9166bf0aca28 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:23.309701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958ad2f-fc73-4bc8-9e9a-9166bf0aca28 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:23.309978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958ad2f-fc73-4bc8-9e9a-9166bf0aca28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:23.319896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958ad2f-fc73-4bc8-9e9a-9166bf0aca28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-0958ad2f-fc73-4bc8-9e9a-9166bf0aca28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:25:23.328803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:23.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:23.330817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:23.330817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:23.330817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:23.330817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:23.330817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:25:23.330817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:25:23.330817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:25:23.330817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:25:23.330817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:25:23.330817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:25:23.330817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:25:23.330817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:23.330817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 637/1276] 10.222.0.64 () {56 vars in 1096 bytes} [Mon May 23 15:25:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:23.336022 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb369f-c8dc-407e-b6cf-83496bf5bcc6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:23.416956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:23.434286 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb369f-c8dc-407e-b6cf-83496bf5bcc6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:23.467248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21ec74-2bd4-4b32-8e7e-068d3c1702c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:23.467919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21ec74-2bd4-4b32-8e7e-068d3c1702c3 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:23.468203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21ec74-2bd4-4b32-8e7e-068d3c1702c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:23.468478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21ec74-2bd4-4b32-8e7e-068d3c1702c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:23.469295 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21ec74-2bd4-4b32-8e7e-068d3c1702c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:23.469603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21ec74-2bd4-4b32-8e7e-068d3c1702c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:23.470964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21ec74-2bd4-4b32-8e7e-068d3c1702c3 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:23.478825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 640/1277] 10.222.0.64 () {56 vars in 1099 bytes} [Mon May 23 15:25:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:23.484867 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958ad2f-fc73-4bc8-9e9a-9166bf0aca28 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:23.583613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958ad2f-fc73-4bc8-9e9a-9166bf0aca28 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:23.616586 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8d47c-f0e3-4ba0-9be4-14929d0c4957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:23.617400 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8d47c-f0e3-4ba0-9be4-14929d0c4957 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:23.617690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8d47c-f0e3-4ba0-9be4-14929d0c4957 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:23.617971 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8d47c-f0e3-4ba0-9be4-14929d0c4957 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:23.628503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8d47c-f0e3-4ba0-9be4-14929d0c4957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:23.629972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8d47c-f0e3-4ba0-9be4-14929d0c4957 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:23.647015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f1f8d47c-f0e3-4ba0-9be4-14929d0c4957 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:23.647015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:23.647015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:23.647015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:23.647015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:23.647015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:23.647015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:23.647015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:23.647015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:23.647015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:23.647015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:23.647015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:23.647015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:25:23.647015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:25:23.647015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:23.648160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:23.649167 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:25:23.649167 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:25:23.649167 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:23.649167 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:23.649167 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 638/1278] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:23.655237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21ec74-2bd4-4b32-8e7e-068d3c1702c3 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:23.755055 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21ec74-2bd4-4b32-8e7e-068d3c1702c3 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:23.788701 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16957b53-882d-4473-bdfd-f1d040ce4772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:23.789389 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16957b53-882d-4473-bdfd-f1d040ce4772 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:23.789677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16957b53-882d-4473-bdfd-f1d040ce4772 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:23.789958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16957b53-882d-4473-bdfd-f1d040ce4772 None admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:23.790681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16957b53-882d-4473-bdfd-f1d040ce4772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:23.790982 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16957b53-882d-4473-bdfd-f1d040ce4772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:23.792347 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16957b53-882d-4473-bdfd-f1d040ce4772 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:23.803944 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 641/1279] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:23.809858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8d47c-f0e3-4ba0-9be4-14929d0c4957 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:23.841527 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:23.909065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8d47c-f0e3-4ba0-9be4-14929d0c4957 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:23.943737 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdcb455-0e1f-4e6b-a043-3f9537352bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:23.944542 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcb455-0e1f-4e6b-a043-3f9537352bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:23.944831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcb455-0e1f-4e6b-a043-3f9537352bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:23.945148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcb455-0e1f-4e6b-a043-3f9537352bf5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:23.954279 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdcb455-0e1f-4e6b-a043-3f9537352bf5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:23.955717 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdcb455-0e1f-4e6b-a043-3f9537352bf5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-bfdcb455-0e1f-4e6b-a043-3f9537352bf5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:25:23.970049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:23.971008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:23.972212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:23.972212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:23.972212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:25:23.972212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:25:23.972212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:25:23.972212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:25:23.972212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:25:23.972212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:25:23.972212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:25:23.972212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:23.972212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 639/1280] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:23.977351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16957b53-882d-4473-bdfd-f1d040ce4772 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:23.982111 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:24.076771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16957b53-882d-4473-bdfd-f1d040ce4772 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:24.112201 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f3fe0-ea46-4250-8a58-ab4c501a6429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:24.112805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f3fe0-ea46-4250-8a58-ab4c501a6429 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:24.113125 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f3fe0-ea46-4250-8a58-ab4c501a6429 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:24.113408 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f3fe0-ea46-4250-8a58-ab4c501a6429 None admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:24.114236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f3fe0-ea46-4250-8a58-ab4c501a6429 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:24.114549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f3fe0-ea46-4250-8a58-ab4c501a6429 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:24.116012 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f3fe0-ea46-4250-8a58-ab4c501a6429 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:24.132602 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 642/1281] 10.222.0.64 () {56 vars in 1143 bytes} [Mon May 23 15:25:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:24.138563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdcb455-0e1f-4e6b-a043-3f9537352bf5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:24.227823 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:24.237104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdcb455-0e1f-4e6b-a043-3f9537352bf5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:24.270510 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e287620-7987-46ff-95a4-f1293379bbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:24.271155 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e287620-7987-46ff-95a4-f1293379bbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:24.271440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e287620-7987-46ff-95a4-f1293379bbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:24.271719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e287620-7987-46ff-95a4-f1293379bbe7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:24.339344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e287620-7987-46ff-95a4-f1293379bbe7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:24.340799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e287620-7987-46ff-95a4-f1293379bbe7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:24.425585 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:24.435832 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 640/1282] 10.222.0.64 () {58 vars in 1282 bytes} [Mon May 23 15:25:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 15:25:24.441760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f3fe0-ea46-4250-8a58-ab4c501a6429 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:24.502837 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:24.539960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f3fe0-ea46-4250-8a58-ab4c501a6429 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:24.574092 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664e208-1dfd-43e5-9685-3fc0c9c35876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:24.574881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664e208-1dfd-43e5-9685-3fc0c9c35876 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:24.575169 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664e208-1dfd-43e5-9685-3fc0c9c35876 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:24.575450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664e208-1dfd-43e5-9685-3fc0c9c35876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:24.586492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664e208-1dfd-43e5-9685-3fc0c9c35876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:24.589326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664e208-1dfd-43e5-9685-3fc0c9c35876 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:24.606140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-0664e208-1dfd-43e5-9685-3fc0c9c35876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:24.606140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:24.606140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:24.606140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:24.606140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:24.606140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:24.606140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:24.606140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:24.606140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:24.606140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:24.606140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:24.606140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:24.606140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:25:24.606140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:25:24.606140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:24.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:24.608262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:25:24.608262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:25:24.608262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:25:24.608262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:25:24.608262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 643/1283] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:25:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:24.613216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e287620-7987-46ff-95a4-f1293379bbe7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:24.733202 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e287620-7987-46ff-95a4-f1293379bbe7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:24.766761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59abc1-7bb8-4c47-ae39-9da30c55a71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:24.767428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59abc1-7bb8-4c47-ae39-9da30c55a71b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:24.767732 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59abc1-7bb8-4c47-ae39-9da30c55a71b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:24.768013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59abc1-7bb8-4c47-ae39-9da30c55a71b None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:24.768726 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59abc1-7bb8-4c47-ae39-9da30c55a71b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:24.769038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59abc1-7bb8-4c47-ae39-9da30c55a71b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:24.770425 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59abc1-7bb8-4c47-ae39-9da30c55a71b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:24.781500 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 641/1284] 10.222.0.64 () {56 vars in 1109 bytes} [Mon May 23 15:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:24.787144 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664e208-1dfd-43e5-9685-3fc0c9c35876 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:24.884566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664e208-1dfd-43e5-9685-3fc0c9c35876 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:24.897876 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:24.917852 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd473dce-d851-4fec-8503-a04dbc1a2266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:24.918646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd473dce-d851-4fec-8503-a04dbc1a2266 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:24.918933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd473dce-d851-4fec-8503-a04dbc1a2266 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:24.919217 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd473dce-d851-4fec-8503-a04dbc1a2266 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:24.926025 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd473dce-d851-4fec-8503-a04dbc1a2266 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:24.928875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd473dce-d851-4fec-8503-a04dbc1a2266 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:24.941771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application [None req-bd473dce-d851-4fec-8503-a04dbc1a2266 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:25:24.941771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:24.941771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:24.941771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:24.941771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:24.941771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:24.941771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:24.941771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:24.941771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:24.941771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:24.941771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:24.941771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:24.941771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:25:24.941771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:25:24.941771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application with Lock( May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:24.942859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:24.943853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:25:24.943853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:25:24.943853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:25:24.943853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:25:24.943853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:25:24.943853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:25:24.943853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:25:24.943853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: ERROR keystone.server.flask.application May 23 15:25:24.943853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 644/1285] 10.222.0.64 () {56 vars in 1111 bytes} [Mon May 23 15:25:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:24.948961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59abc1-7bb8-4c47-ae39-9da30c55a71b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:24.998415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:25.046692 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59abc1-7bb8-4c47-ae39-9da30c55a71b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:25.080557 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b1988-7b90-4b04-95b5-6231039669ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:25.081439 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b1988-7b90-4b04-95b5-6231039669ef None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:25.085086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b1988-7b90-4b04-95b5-6231039669ef None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:25.085086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b1988-7b90-4b04-95b5-6231039669ef None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:25.085086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88b1988-7b90-4b04-95b5-6231039669ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:25.085086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88b1988-7b90-4b04-95b5-6231039669ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:25.086420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88b1988-7b90-4b04-95b5-6231039669ef None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:25.100077 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 642/1286] 10.222.0.64 () {56 vars in 1148 bytes} [Mon May 23 15:25:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:25.107204 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd473dce-d851-4fec-8503-a04dbc1a2266 None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:25.205983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd473dce-d851-4fec-8503-a04dbc1a2266 None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:25.240240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f8e9c-34cc-4182-ad66-39467b7c9181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:25.240909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f8e9c-34cc-4182-ad66-39467b7c9181 None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:25.241252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f8e9c-34cc-4182-ad66-39467b7c9181 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:25.241529 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f8e9c-34cc-4182-ad66-39467b7c9181 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:25.242390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f8e9c-34cc-4182-ad66-39467b7c9181 None admin] RBAC: Adding query filter params (role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e, scope.project.id=d8d81eafac474f37a71ee5cc292d172f, user.id=04fbaeb7583a456d81cf79f6b101fb1b) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:25.242702 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f8e9c-34cc-4182-ad66-39467b7c9181 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:25.244451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f8e9c-34cc-4182-ad66-39467b7c9181 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:25.250263 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 645/1287] 10.222.0.64 () {56 vars in 1374 bytes} [Mon May 23 15:25:25 2022] GET /identity/v3/role_assignments?user.id=04fbaeb7583a456d81cf79f6b101fb1b&scope.project.id=d8d81eafac474f37a71ee5cc292d172f&role.id=cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:27.518997 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec407b7-7c99-49ec-bc4a-69be38578317 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:27.519582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec407b7-7c99-49ec-bc4a-69be38578317 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:27.519582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec407b7-7c99-49ec-bc4a-69be38578317 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:27.520999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 643/1288] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:25:27.526236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc6e27-097d-45b8-87d0-894f332aeab3 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:27.526646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc6e27-097d-45b8-87d0-894f332aeab3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:27.526796 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc6e27-097d-45b8-87d0-894f332aeab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:27.543895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-02dc6e27-097d-45b8-87d0-894f332aeab3 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:25:27.545096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-02dc6e27-097d-45b8-87d0-894f332aeab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:27.574112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:27.610022 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 646/1289] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:25:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:25:27.632429 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec407b7-7c99-49ec-bc4a-69be38578317 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:27.748148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec407b7-7c99-49ec-bc4a-69be38578317 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:27.759435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:27.781157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844a2fb-e8df-4c48-bcaa-6d55b5a10117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:27.781950 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844a2fb-e8df-4c48-bcaa-6d55b5a10117 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:27.782231 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844a2fb-e8df-4c48-bcaa-6d55b5a10117 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:27.782503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844a2fb-e8df-4c48-bcaa-6d55b5a10117 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:27.783153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844a2fb-e8df-4c48-bcaa-6d55b5a10117 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:27.784531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844a2fb-e8df-4c48-bcaa-6d55b5a10117 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-7844a2fb-e8df-4c48-bcaa-6d55b5a10117 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:27.796482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:27.797773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:27.798884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:27.798884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:27.798884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:27.798884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:27.798884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:27.798884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:27.798884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:27.798884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:27.798884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:27.798884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:27.798884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:25:27.798884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:25:27.798884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:25:27.798884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:27.798884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 644/1290] 10.222.0.64 () {56 vars in 1117 bytes} [Mon May 23 15:25:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:27.804147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc6e27-097d-45b8-87d0-894f332aeab3 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:27.906186 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc6e27-097d-45b8-87d0-894f332aeab3 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:27.939265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13980611-09f8-4ce9-a17f-e862377b3e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:27.939922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13980611-09f8-4ce9-a17f-e862377b3e9b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:27.940205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13980611-09f8-4ce9-a17f-e862377b3e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:27.940481 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13980611-09f8-4ce9-a17f-e862377b3e9b None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:27.941206 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13980611-09f8-4ce9-a17f-e862377b3e9b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:27.941514 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13980611-09f8-4ce9-a17f-e862377b3e9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:27.942883 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13980611-09f8-4ce9-a17f-e862377b3e9b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:27.947649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 647/1291] 10.222.0.64 () {56 vars in 1116 bytes} [Mon May 23 15:25:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:27.953421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844a2fb-e8df-4c48-bcaa-6d55b5a10117 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:28.051250 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844a2fb-e8df-4c48-bcaa-6d55b5a10117 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:28.082868 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df87781-7853-4a87-8c31-44a65cbad0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:28.083523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df87781-7853-4a87-8c31-44a65cbad0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:28.083812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df87781-7853-4a87-8c31-44a65cbad0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:28.084091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df87781-7853-4a87-8c31-44a65cbad0a2 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:28.084786 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df87781-7853-4a87-8c31-44a65cbad0a2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:28.085141 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df87781-7853-4a87-8c31-44a65cbad0a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:28.086513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df87781-7853-4a87-8c31-44a65cbad0a2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:28.091368 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 645/1292] 10.222.0.64 () {56 vars in 1116 bytes} [Mon May 23 15:25:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:30.281174 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cfeab-345d-4895-b163-57f5c8166be3 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:30.281714 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cfeab-345d-4895-b163-57f5c8166be3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:30.281808 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cfeab-345d-4895-b163-57f5c8166be3 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:30.283110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 648/1293] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:25:30.288182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60bf79-4149-447b-91b8-e67a158e3053 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:30.288581 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60bf79-4149-447b-91b8-e67a158e3053 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:30.288746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60bf79-4149-447b-91b8-e67a158e3053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:30.305889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ab60bf79-4149-447b-91b8-e67a158e3053 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:25:30.307176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ab60bf79-4149-447b-91b8-e67a158e3053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:30.335291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:30.370812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 646/1294] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:25:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:25:30.393413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cfeab-345d-4895-b163-57f5c8166be3 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:30.426291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:30.495775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cfeab-345d-4895-b163-57f5c8166be3 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:30.529822 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9dafc-19e9-4197-9f77-b59cb877d961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:30.530486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9dafc-19e9-4197-9f77-b59cb877d961 None admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:30.530776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9dafc-19e9-4197-9f77-b59cb877d961 None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:30.531142 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9dafc-19e9-4197-9f77-b59cb877d961 None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:30.531912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b9dafc-19e9-4197-9f77-b59cb877d961 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:30.533354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b9dafc-19e9-4197-9f77-b59cb877d961 None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:30.607349 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 649/1295] 10.222.0.64 () {60 vars in 1141 bytes} [Mon May 23 15:25:30 2022] POST /identity/v3/services => generated 246 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:25:32.862224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0456fe3-6c3c-42ec-a716-3ba8b1ea3662 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:32.862773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0456fe3-6c3c-42ec-a716-3ba8b1ea3662 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:32.862773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0456fe3-6c3c-42ec-a716-3ba8b1ea3662 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:32.864166 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 647/1296] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:25:32.869947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357945e8-4747-4f9c-8d37-83e7347d6ac4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:32.870358 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357945e8-4747-4f9c-8d37-83e7347d6ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:32.870521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357945e8-4747-4f9c-8d37-83e7347d6ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:32.887042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-357945e8-4747-4f9c-8d37-83e7347d6ac4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:25:32.888244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-357945e8-4747-4f9c-8d37-83e7347d6ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:32.959806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 650/1297] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:25:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:25:32.982251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0456fe3-6c3c-42ec-a716-3ba8b1ea3662 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:33.079582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0456fe3-6c3c-42ec-a716-3ba8b1ea3662 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:33.112432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cab12-02eb-40e7-b57b-faff3c083bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:33.113274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cab12-02eb-40e7-b57b-faff3c083bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:33.113532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cab12-02eb-40e7-b57b-faff3c083bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:33.113808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cab12-02eb-40e7-b57b-faff3c083bc5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:33.114451 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cab12-02eb-40e7-b57b-faff3c083bc5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:33.115804 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cab12-02eb-40e7-b57b-faff3c083bc5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a05cab12-02eb-40e7-b57b-faff3c083bc5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:33.127495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:33.128782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:33.130287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:33.130287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:33.130287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:33.130287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:33.130287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:33.130287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:33.130287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:33.130287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:33.130287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:33.130287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:33.130287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:25:33.130287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:25:33.130287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:25:33.130287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:33.130287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 648/1298] 10.222.0.64 () {56 vars in 1117 bytes} [Mon May 23 15:25:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:33.134743 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357945e8-4747-4f9c-8d37-83e7347d6ac4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:33.153454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:33.229876 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357945e8-4747-4f9c-8d37-83e7347d6ac4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:33.263281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2e9e0-3a18-4f38-ab3c-840f29fb807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:33.263954 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2e9e0-3a18-4f38-ab3c-840f29fb807d None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:33.264239 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2e9e0-3a18-4f38-ab3c-840f29fb807d None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:33.264520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2e9e0-3a18-4f38-ab3c-840f29fb807d None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:33.265257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2e9e0-3a18-4f38-ab3c-840f29fb807d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:33.265575 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2e9e0-3a18-4f38-ab3c-840f29fb807d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:33.266950 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2e9e0-3a18-4f38-ab3c-840f29fb807d None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:33.272734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 651/1299] 10.222.0.64 () {56 vars in 1116 bytes} [Mon May 23 15:25:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:33.279003 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cab12-02eb-40e7-b57b-faff3c083bc5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:33.311131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:33.377914 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cab12-02eb-40e7-b57b-faff3c083bc5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:33.410317 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0c815-01e2-42fb-881e-54cc83ca7525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:33.410966 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0c815-01e2-42fb-881e-54cc83ca7525 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:33.411252 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0c815-01e2-42fb-881e-54cc83ca7525 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:33.411529 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0c815-01e2-42fb-881e-54cc83ca7525 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:33.412310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0c815-01e2-42fb-881e-54cc83ca7525 None admin] RBAC: Adding query filter params (interface=public, service_id=70e01ec94f2a49aa884e4e78e6d78ed3, region_id=RegionOne) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:33.412623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0c815-01e2-42fb-881e-54cc83ca7525 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:33.414005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0c815-01e2-42fb-881e-54cc83ca7525 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:33.419595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 649/1300] 10.222.0.64 () {56 vars in 1251 bytes} [Mon May 23 15:25:33 2022] GET /identity/v3/endpoints?service_id=70e01ec94f2a49aa884e4e78e6d78ed3&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:33.425558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2e9e0-3a18-4f38-ab3c-840f29fb807d None admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:33.505431 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:33.523234 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2e9e0-3a18-4f38-ab3c-840f29fb807d None admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:33.556684 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74265177-78cd-4987-abcd-6314cc48c62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:33.557373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74265177-78cd-4987-abcd-6314cc48c62c None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:33.557658 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74265177-78cd-4987-abcd-6314cc48c62c None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:33.557935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74265177-78cd-4987-abcd-6314cc48c62c None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:33.558610 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74265177-78cd-4987-abcd-6314cc48c62c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:33.559984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74265177-78cd-4987-abcd-6314cc48c62c None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:33.566956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 652/1301] 10.222.0.64 () {56 vars in 1087 bytes} [Mon May 23 15:25:33 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:25:35.762391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed2b67-6ca5-4a4a-bcf5-2d30ba26fcbe None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:35.762976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed2b67-6ca5-4a4a-bcf5-2d30ba26fcbe None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:35.762976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed2b67-6ca5-4a4a-bcf5-2d30ba26fcbe None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:35.764334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 650/1302] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:25:35.769711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da34a12-a549-43bb-a9e7-228a0a37bf47 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:35.770107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da34a12-a549-43bb-a9e7-228a0a37bf47 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:35.770279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da34a12-a549-43bb-a9e7-228a0a37bf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:35.787967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8da34a12-a549-43bb-a9e7-228a0a37bf47 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:25:35.789186 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8da34a12-a549-43bb-a9e7-228a0a37bf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:25:35.855654 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 653/1303] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:25:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:25:35.878292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed2b67-6ca5-4a4a-bcf5-2d30ba26fcbe None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:35.972376 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:35.978985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed2b67-6ca5-4a4a-bcf5-2d30ba26fcbe None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:36.011240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57abb0-055f-4e04-a69d-9cd434d70c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:36.012035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57abb0-055f-4e04-a69d-9cd434d70c41 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:36.012323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57abb0-055f-4e04-a69d-9cd434d70c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:36.012601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57abb0-055f-4e04-a69d-9cd434d70c41 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:36.013282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57abb0-055f-4e04-a69d-9cd434d70c41 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:36.014634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57abb0-055f-4e04-a69d-9cd434d70c41 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-ad57abb0-055f-4e04-a69d-9cd434d70c41 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:25:36.027165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:25:36.034536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:25:36.035613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:25:36.035613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( May 23 15:25:36.035613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:25:36.035613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() May 23 15:25:36.035613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:25:36.035613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:25:36.035613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:25:36.035613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() May 23 15:25:36.035613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:25:36.035613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( May 23 15:25:36.035613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:25:36.035613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:25:36.035613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:25:36.035613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application May 23 15:25:36.035613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 651/1304] 10.222.0.64 () {56 vars in 1117 bytes} [Mon May 23 15:25:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:25:36.036688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da34a12-a549-43bb-a9e7-228a0a37bf47 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:36.109304 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:36.131167 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da34a12-a549-43bb-a9e7-228a0a37bf47 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:36.163620 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b59449-1fe8-4262-b88e-42ad3fe94e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:36.164283 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b59449-1fe8-4262-b88e-42ad3fe94e2b None admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:36.164575 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b59449-1fe8-4262-b88e-42ad3fe94e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:36.164856 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b59449-1fe8-4262-b88e-42ad3fe94e2b None admin] PATH_INFO: `/v3/services` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:36.165625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b59449-1fe8-4262-b88e-42ad3fe94e2b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:25:36.166049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b59449-1fe8-4262-b88e-42ad3fe94e2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:36.167555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b59449-1fe8-4262-b88e-42ad3fe94e2b None admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:36.172499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 654/1305] 10.222.0.64 () {56 vars in 1116 bytes} [Mon May 23 15:25:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:25:36.178555 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57abb0-055f-4e04-a69d-9cd434d70c41 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:25:36.277569 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57abb0-055f-4e04-a69d-9cd434d70c41 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:25:36.310413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd188c4e-b386-43e4-a9c4-828131e4f4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:25:36.311066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd188c4e-b386-43e4-a9c4-828131e4f4d8 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:25:36.311355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd188c4e-b386-43e4-a9c4-828131e4f4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:25:36.311642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd188c4e-b386-43e4-a9c4-828131e4f4d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:25:36.312382 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd188c4e-b386-43e4-a9c4-828131e4f4d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:25:36.313809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd188c4e-b386-43e4-a9c4-828131e4f4d8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:25:36.413740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 652/1306] 10.222.0.64 () {60 vars in 1144 bytes} [Mon May 23 15:25:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:25:46.263980 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=30 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:25:46.409184 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:24.736137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d081408-494d-4e1c-b04e-6f8958d63a35 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:24.736572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d081408-494d-4e1c-b04e-6f8958d63a35 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:24.736781 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d081408-494d-4e1c-b04e-6f8958d63a35 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:24.740542 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 655/1307] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:32:24.743372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fe66b-2279-47eb-adba-1eded3faf0c7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:24.743811 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fe66b-2279-47eb-adba-1eded3faf0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:24.743958 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fe66b-2279-47eb-adba-1eded3faf0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:24.836631 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-820fe66b-2279-47eb-adba-1eded3faf0c7 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:24.837887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-820fe66b-2279-47eb-adba-1eded3faf0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:25.000219 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 653/1308] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:32:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:32:25.039660 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16319549-ed8f-4d29-b45a-49d50ef1be36 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:25.040330 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16319549-ed8f-4d29-b45a-49d50ef1be36 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:25.040786 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16319549-ed8f-4d29-b45a-49d50ef1be36 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:25.042528 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 656/1309] 10.222.0.64 () {54 vars in 964 bytes} [Mon May 23 15:32:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:32:25.053493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85a4e4-8612-4874-a25e-11c1916e5ddc None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:25.053493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85a4e4-8612-4874-a25e-11c1916e5ddc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:25.053493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85a4e4-8612-4874-a25e-11c1916e5ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:25.137940 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-cf85a4e4-8612-4874-a25e-11c1916e5ddc None None] MFA Rules not processed for user `12ceb8a355a44b9fb29918483379cb38`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:25.140069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cf85a4e4-8612-4874-a25e-11c1916e5ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:25.163478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:25.229607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 654/1310] 10.222.0.64 () {58 vars in 1061 bytes} [Mon May 23 15:32:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:25.239078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16319549-ed8f-4d29-b45a-49d50ef1be36 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:25.240774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-16319549-ed8f-4d29-b45a-49d50ef1be36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:25.363029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16319549-ed8f-4d29-b45a-49d50ef1be36 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:25.443104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e456e-f24a-4d65-9635-ebdd423c080c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:25.443775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e456e-f24a-4d65-9635-ebdd423c080c service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:25.444118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e456e-f24a-4d65-9635-ebdd423c080c service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:25.444327 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e456e-f24a-4d65-9635-ebdd423c080c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:25.446052 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c10e456e-f24a-4d65-9635-ebdd423c080c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:25.507715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:25.563973 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e456e-f24a-4d65-9635-ebdd423c080c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:25.565493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e456e-f24a-4d65-9635-ebdd423c080c service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:25.587486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 657/1311] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:32:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:28.109011 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2166ea-a19d-4d8f-bb8d-5e9c7b752182 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:28.109573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2166ea-a19d-4d8f-bb8d-5e9c7b752182 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:28.109573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2166ea-a19d-4d8f-bb8d-5e9c7b752182 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:28.110911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 655/1312] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:32:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:32:28.116303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa61a0-2e72-4d8a-a848-8023ad993314 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:28.116800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa61a0-2e72-4d8a-a848-8023ad993314 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:28.116909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa61a0-2e72-4d8a-a848-8023ad993314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:28.192045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a3fa61a0-2e72-4d8a-a848-8023ad993314 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:28.193332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a3fa61a0-2e72-4d8a-a848-8023ad993314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:28.304667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 658/1313] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:28.343361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2166ea-a19d-4d8f-bb8d-5e9c7b752182 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:28.344881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ea2166ea-a19d-4d8f-bb8d-5e9c7b752182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:28.448815 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2166ea-a19d-4d8f-bb8d-5e9c7b752182 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:28.522422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:28.525036 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa16d3-b01f-48e5-a505-78fb04cd80d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:28.525711 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa16d3-b01f-48e5-a505-78fb04cd80d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:28.525996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa16d3-b01f-48e5-a505-78fb04cd80d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:28.526272 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa16d3-b01f-48e5-a505-78fb04cd80d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:28.527922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8faa16d3-b01f-48e5-a505-78fb04cd80d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:28.638193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa16d3-b01f-48e5-a505-78fb04cd80d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:28.639571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa16d3-b01f-48e5-a505-78fb04cd80d3 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:28.657580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 656/1314] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:32:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:31.470311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87851c93-d8a0-4f12-883f-5545b2fd8bd7 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:31.470845 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87851c93-d8a0-4f12-883f-5545b2fd8bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:31.470938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87851c93-d8a0-4f12-883f-5545b2fd8bd7 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:31.472255 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 659/1315] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:32:31.477450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a3828-1497-40a5-9b30-14bf5b135573 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:31.477862 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a3828-1497-40a5-9b30-14bf5b135573 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:31.478030 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a3828-1497-40a5-9b30-14bf5b135573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:31.541968 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0b0a3828-1497-40a5-9b30-14bf5b135573 None None] MFA Rules not processed for user `358e903ec0d14cd2809ef65ffb8035f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:31.543183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0b0a3828-1497-40a5-9b30-14bf5b135573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:31.596438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:31.642733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 657/1316] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:32:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:31.682135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87851c93-d8a0-4f12-883f-5545b2fd8bd7 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:31.683664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-87851c93-d8a0-4f12-883f-5545b2fd8bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:31.693823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:31.795639 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87851c93-d8a0-4f12-883f-5545b2fd8bd7 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:31.872006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b43fa9-7144-4633-9f6d-6b78cd033607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:31.872665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b43fa9-7144-4633-9f6d-6b78cd033607 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:31.873012 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b43fa9-7144-4633-9f6d-6b78cd033607 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:31.873324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b43fa9-7144-4633-9f6d-6b78cd033607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:31.874963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-48b43fa9-7144-4633-9f6d-6b78cd033607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:31.938810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:31.991242 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b43fa9-7144-4633-9f6d-6b78cd033607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:31.992719 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b43fa9-7144-4633-9f6d-6b78cd033607 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:32.012781 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 660/1317] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:32:31 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:35.037631 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb749193-342c-4011-81f8-bed5da9b465f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:35.038153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb749193-342c-4011-81f8-bed5da9b465f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:35.038265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb749193-342c-4011-81f8-bed5da9b465f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:35.039563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 658/1318] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:32:35.045472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcffedf-26ae-4f82-b4c2-2d0a8930a88f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:35.045911 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcffedf-26ae-4f82-b4c2-2d0a8930a88f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:35.046018 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcffedf-26ae-4f82-b4c2-2d0a8930a88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:35.110448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ebcffedf-26ae-4f82-b4c2-2d0a8930a88f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:35.111671 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ebcffedf-26ae-4f82-b4c2-2d0a8930a88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:35.207577 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 661/1319] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:32:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:32:35.247112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb749193-342c-4011-81f8-bed5da9b465f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:35.248751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-eb749193-342c-4011-81f8-bed5da9b465f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:35.350789 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb749193-342c-4011-81f8-bed5da9b465f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:35.422061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c667f1e-ca06-47b5-a047-fed8c87afe24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:35.422717 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c667f1e-ca06-47b5-a047-fed8c87afe24 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:35.423007 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c667f1e-ca06-47b5-a047-fed8c87afe24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:35.423286 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c667f1e-ca06-47b5-a047-fed8c87afe24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:35.424918 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2c667f1e-ca06-47b5-a047-fed8c87afe24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:35.432390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:35.531649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c667f1e-ca06-47b5-a047-fed8c87afe24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:35.533120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c667f1e-ca06-47b5-a047-fed8c87afe24 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:35.551003 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 659/1320] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:32:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:32:38.800605 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99106688-fdde-45ba-a37f-d3930e5f83e5 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:38.801134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99106688-fdde-45ba-a37f-d3930e5f83e5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:38.801245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99106688-fdde-45ba-a37f-d3930e5f83e5 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:38.802816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 662/1321] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:32:38.808057 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75272c-b73e-472b-9845-fa9b854166d5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:38.808461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75272c-b73e-472b-9845-fa9b854166d5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:38.808751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75272c-b73e-472b-9845-fa9b854166d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:38.874256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9c75272c-b73e-472b-9845-fa9b854166d5 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:38.875468 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9c75272c-b73e-472b-9845-fa9b854166d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:38.932713 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:38.972312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 660/1322] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:39.007858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99106688-fdde-45ba-a37f-d3930e5f83e5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:39.009514 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-99106688-fdde-45ba-a37f-d3930e5f83e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:39.029516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:39.111677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99106688-fdde-45ba-a37f-d3930e5f83e5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:39.183660 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafffba-3e08-4163-b908-da6d68f9b49d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:39.184313 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafffba-3e08-4163-b908-da6d68f9b49d service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:39.184604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafffba-3e08-4163-b908-da6d68f9b49d service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:39.184890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafffba-3e08-4163-b908-da6d68f9b49d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:39.187296 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bfafffba-3e08-4163-b908-da6d68f9b49d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:39.257543 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:39.295634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafffba-3e08-4163-b908-da6d68f9b49d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:39.297599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafffba-3e08-4163-b908-da6d68f9b49d service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:39.316711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 663/1323] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:32:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:41.586385 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a0e14-49a1-41a9-add2-6d89072d0516 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:41.586960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a0e14-49a1-41a9-add2-6d89072d0516 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:41.586960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a0e14-49a1-41a9-add2-6d89072d0516 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:41.588299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 661/1324] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:32:41.593962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28741e-dbf9-46a9-8c76-b91cf5494d51 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:41.594393 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28741e-dbf9-46a9-8c76-b91cf5494d51 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:41.594614 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28741e-dbf9-46a9-8c76-b91cf5494d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:41.665702 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-de28741e-dbf9-46a9-8c76-b91cf5494d51 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:41.666914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-de28741e-dbf9-46a9-8c76-b91cf5494d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:41.766082 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 664/1325] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:41.802387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a0e14-49a1-41a9-add2-6d89072d0516 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:41.803899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-916a0e14-49a1-41a9-add2-6d89072d0516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:41.906647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a0e14-49a1-41a9-add2-6d89072d0516 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:41.980439 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70fe1e-a34b-4d36-a4ea-5eae8cf052bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:41.981115 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70fe1e-a34b-4d36-a4ea-5eae8cf052bb service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:41.981403 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70fe1e-a34b-4d36-a4ea-5eae8cf052bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:41.981679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70fe1e-a34b-4d36-a4ea-5eae8cf052bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:41.983552 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6d70fe1e-a34b-4d36-a4ea-5eae8cf052bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:42.000504 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:42.094799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70fe1e-a34b-4d36-a4ea-5eae8cf052bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:42.096238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70fe1e-a34b-4d36-a4ea-5eae8cf052bb service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:42.114846 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 662/1326] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:32:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:44.458736 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfb265-b74d-4b86-b08f-2b4c72cc9af6 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:44.459308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfb265-b74d-4b86-b08f-2b4c72cc9af6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:44.459308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfb265-b74d-4b86-b08f-2b4c72cc9af6 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:44.460677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 665/1327] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:32:44.465887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf01e8-344f-42bb-85a3-11c34e7e704b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:44.466536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf01e8-344f-42bb-85a3-11c34e7e704b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:44.466824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf01e8-344f-42bb-85a3-11c34e7e704b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:44.531796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-acdf01e8-344f-42bb-85a3-11c34e7e704b None None] MFA Rules not processed for user `358e903ec0d14cd2809ef65ffb8035f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:44.533016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-acdf01e8-344f-42bb-85a3-11c34e7e704b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:44.597909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:44.630563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 663/1328] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:32:44 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:44.666949 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfb265-b74d-4b86-b08f-2b4c72cc9af6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:44.668493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b7cfb265-b74d-4b86-b08f-2b4c72cc9af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:44.701478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:44.774536 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfb265-b74d-4b86-b08f-2b4c72cc9af6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:44.846868 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070fa54c-9e62-4ca2-9721-84158599647f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:44.847523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fa54c-9e62-4ca2-9721-84158599647f service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:44.847807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fa54c-9e62-4ca2-9721-84158599647f service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:44.848083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fa54c-9e62-4ca2-9721-84158599647f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:44.849774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-070fa54c-9e62-4ca2-9721-84158599647f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:44.927029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:44.959449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070fa54c-9e62-4ca2-9721-84158599647f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:44.960896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070fa54c-9e62-4ca2-9721-84158599647f service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:44.980193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 666/1329] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:32:44 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:47.623410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3f7c7-e832-42c7-98c8-38121507e8ec None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:47.623960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3f7c7-e832-42c7-98c8-38121507e8ec None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:47.623960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3f7c7-e832-42c7-98c8-38121507e8ec None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:47.625422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 664/1330] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:32:47.630880 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a8b10-8c31-4425-a81f-aa691383e29d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:47.631306 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a8b10-8c31-4425-a81f-aa691383e29d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:47.631447 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a8b10-8c31-4425-a81f-aa691383e29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:47.697465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-780a8b10-8c31-4425-a81f-aa691383e29d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:47.698877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-780a8b10-8c31-4425-a81f-aa691383e29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:47.794837 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 667/1331] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:32:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:47.831210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3f7c7-e832-42c7-98c8-38121507e8ec None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:47.832759 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c1a3f7c7-e832-42c7-98c8-38121507e8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:47.938276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3f7c7-e832-42c7-98c8-38121507e8ec None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:48.018231 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0862f2-b9ba-41a2-abfa-78f482c70b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:48.018895 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0862f2-b9ba-41a2-abfa-78f482c70b7d service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:48.019190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0862f2-b9ba-41a2-abfa-78f482c70b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:48.019468 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0862f2-b9ba-41a2-abfa-78f482c70b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:48.021334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8e0862f2-b9ba-41a2-abfa-78f482c70b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:48.054306 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:48.148892 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0862f2-b9ba-41a2-abfa-78f482c70b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:48.150367 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0862f2-b9ba-41a2-abfa-78f482c70b7d service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:48.169958 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 665/1332] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:32:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:50.415763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec303e3f-7fac-4e8e-84ec-367f9f48be47 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:50.416314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec303e3f-7fac-4e8e-84ec-367f9f48be47 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:50.416314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec303e3f-7fac-4e8e-84ec-367f9f48be47 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:50.417752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 668/1333] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:32:50.423416 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3947064-42cf-4887-b5be-4be3cd5dfacd None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:50.423826 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3947064-42cf-4887-b5be-4be3cd5dfacd None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:50.423988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3947064-42cf-4887-b5be-4be3cd5dfacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:50.487798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e3947064-42cf-4887-b5be-4be3cd5dfacd None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:50.489007 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e3947064-42cf-4887-b5be-4be3cd5dfacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:50.561891 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:50.584146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 666/1334] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:50.625516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec303e3f-7fac-4e8e-84ec-367f9f48be47 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:50.627052 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ec303e3f-7fac-4e8e-84ec-367f9f48be47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:50.668658 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:50.733959 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec303e3f-7fac-4e8e-84ec-367f9f48be47 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:50.806682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf531a7-25d2-4331-9162-6483f0ff0c59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:50.807342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf531a7-25d2-4331-9162-6483f0ff0c59 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:50.807628 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf531a7-25d2-4331-9162-6483f0ff0c59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:50.807908 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf531a7-25d2-4331-9162-6483f0ff0c59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:50.810161 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fbf531a7-25d2-4331-9162-6483f0ff0c59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:50.893661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:50.919197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf531a7-25d2-4331-9162-6483f0ff0c59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:50.920690 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf531a7-25d2-4331-9162-6483f0ff0c59 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:50.939900 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 669/1335] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:32:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:53.805623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf07bc3-0f76-4cb0-8e80-591d57a8de4f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:53.806151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf07bc3-0f76-4cb0-8e80-591d57a8de4f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:53.806348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf07bc3-0f76-4cb0-8e80-591d57a8de4f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:53.807554 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 667/1336] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:32:53.813181 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0b8bb-62d5-435d-a2b4-ffb204d0d849 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:53.813617 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0b8bb-62d5-435d-a2b4-ffb204d0d849 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:53.813746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0b8bb-62d5-435d-a2b4-ffb204d0d849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:53.882314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-05d0b8bb-62d5-435d-a2b4-ffb204d0d849 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:53.883526 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-05d0b8bb-62d5-435d-a2b4-ffb204d0d849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:53.979728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 670/1337] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:32:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:54.016777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf07bc3-0f76-4cb0-8e80-591d57a8de4f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:54.018331 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-eaf07bc3-0f76-4cb0-8e80-591d57a8de4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:54.124253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf07bc3-0f76-4cb0-8e80-591d57a8de4f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:54.205381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62c3b6-01ef-45c3-b030-6c2f6636121c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:54.206029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62c3b6-01ef-45c3-b030-6c2f6636121c service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:54.206313 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62c3b6-01ef-45c3-b030-6c2f6636121c service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:54.206587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62c3b6-01ef-45c3-b030-6c2f6636121c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:54.208210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3f62c3b6-01ef-45c3-b030-6c2f6636121c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:54.243629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:54.318005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62c3b6-01ef-45c3-b030-6c2f6636121c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:54.319450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62c3b6-01ef-45c3-b030-6c2f6636121c service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:54.339610 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 668/1338] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:32:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:32:58.977161 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e91bd-acba-4928-811b-f3a8f24e90c6 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:58.977704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e91bd-acba-4928-811b-f3a8f24e90c6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:58.977704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e91bd-acba-4928-811b-f3a8f24e90c6 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:58.981935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 671/1339] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:32:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:32:58.985149 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bffc90-f575-4a0b-8c70-50f0ef635ffc None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:58.985612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bffc90-f575-4a0b-8c70-50f0ef635ffc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:58.985910 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bffc90-f575-4a0b-8c70-50f0ef635ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:59.057836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-63bffc90-f575-4a0b-8c70-50f0ef635ffc None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:32:59.059098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-63bffc90-f575-4a0b-8c70-50f0ef635ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:59.166227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:59.180871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 669/1340] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:32:59.219449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e91bd-acba-4928-811b-f3a8f24e90c6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:32:59.221837 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5d0e91bd-acba-4928-811b-f3a8f24e90c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:32:59.272446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:32:59.321497 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e91bd-acba-4928-811b-f3a8f24e90c6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:32:59.325297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121d46b-87a9-49f9-83b5-878c2914191f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:32:59.326089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121d46b-87a9-49f9-83b5-878c2914191f service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:32:59.326381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121d46b-87a9-49f9-83b5-878c2914191f service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:32:59.326705 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121d46b-87a9-49f9-83b5-878c2914191f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:32:59.336623 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121d46b-87a9-49f9-83b5-878c2914191f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:32:59.338386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121d46b-87a9-49f9-83b5-878c2914191f service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:32:59.349647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 672/1341] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:32:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:02.745719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080a8b7-a49e-4e63-ac37-adbbd22cf3b8 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:02.746291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080a8b7-a49e-4e63-ac37-adbbd22cf3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:02.746291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080a8b7-a49e-4e63-ac37-adbbd22cf3b8 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:02.748197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 670/1342] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:02.753522 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780633f1-2c00-4ced-ba3f-1cbe75e604d6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:02.753988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780633f1-2c00-4ced-ba3f-1cbe75e604d6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:02.754221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780633f1-2c00-4ced-ba3f-1cbe75e604d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:02.783258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-780633f1-2c00-4ced-ba3f-1cbe75e604d6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:02.784462 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-780633f1-2c00-4ced-ba3f-1cbe75e604d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:02.864239 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 673/1343] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:02.916936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e2b73-2435-48b0-9bde-bf04b162320f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:02.917314 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e2b73-2435-48b0-9bde-bf04b162320f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:02.917598 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e2b73-2435-48b0-9bde-bf04b162320f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:02.919284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 671/1344] 10.222.0.64 () {54 vars in 964 bytes} [Mon May 23 15:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:02.925592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723b87-3058-451e-83ac-fa356ced50b8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:02.925966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723b87-3058-451e-83ac-fa356ced50b8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:02.926312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723b87-3058-451e-83ac-fa356ced50b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:02.984775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-5e723b87-3058-451e-83ac-fa356ced50b8 None None] MFA Rules not processed for user `12ceb8a355a44b9fb29918483379cb38`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:02.985918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5e723b87-3058-451e-83ac-fa356ced50b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:02.992603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 674/1345] 10.222.0.64 () {58 vars in 1061 bytes} [Mon May 23 15:33:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:03.002339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e2b73-2435-48b0-9bde-bf04b162320f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:03.017020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e2b73-2435-48b0-9bde-bf04b162320f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:03.020681 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d85f04-e140-4072-9fdf-fc8b280a12e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:03.021427 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d85f04-e140-4072-9fdf-fc8b280a12e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:03.021779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d85f04-e140-4072-9fdf-fc8b280a12e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:03.022124 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d85f04-e140-4072-9fdf-fc8b280a12e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:03.031490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d85f04-e140-4072-9fdf-fc8b280a12e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:03.033014 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d85f04-e140-4072-9fdf-fc8b280a12e9 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:03.042476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 672/1346] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:08.419387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a8ae0-d3ac-4d9e-8427-ae92557c4e59 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:08.419955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a8ae0-d3ac-4d9e-8427-ae92557c4e59 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:08.419955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a8ae0-d3ac-4d9e-8427-ae92557c4e59 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:08.421388 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 675/1347] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:08.427197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a84eb9-90f7-4797-bc08-1cc2fb3b2d39 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:08.427658 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a84eb9-90f7-4797-bc08-1cc2fb3b2d39 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:08.427778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a84eb9-90f7-4797-bc08-1cc2fb3b2d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:08.447125 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-96a84eb9-90f7-4797-bc08-1cc2fb3b2d39 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:08.448360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-96a84eb9-90f7-4797-bc08-1cc2fb3b2d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:08.456304 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 673/1348] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:33:08.494376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a8ae0-d3ac-4d9e-8427-ae92557c4e59 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:08.505491 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a8ae0-d3ac-4d9e-8427-ae92557c4e59 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:08.509164 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec3ab1-f7b3-403d-872e-971e2e12a458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:08.509825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec3ab1-f7b3-403d-872e-971e2e12a458 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:08.510116 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec3ab1-f7b3-403d-872e-971e2e12a458 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:08.510391 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec3ab1-f7b3-403d-872e-971e2e12a458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:08.520891 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec3ab1-f7b3-403d-872e-971e2e12a458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:08.522460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec3ab1-f7b3-403d-872e-971e2e12a458 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:08.533335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 676/1349] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:13.964018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b455ebc-74c2-40b6-b787-d8d9109ae029 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:13.964603 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b455ebc-74c2-40b6-b787-d8d9109ae029 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:13.964694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b455ebc-74c2-40b6-b787-d8d9109ae029 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:13.966288 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 674/1350] 10.222.0.64 () {54 vars in 921 bytes} [Mon May 23 15:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:13.971477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ce529-b796-4213-9102-e615e8ec4ad0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:13.971939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ce529-b796-4213-9102-e615e8ec4ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:13.972247 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ce529-b796-4213-9102-e615e8ec4ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:14.026686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-950ce529-b796-4213-9102-e615e8ec4ad0 None None] MFA Rules not processed for user `04fbaeb7583a456d81cf79f6b101fb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:14.029043 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-950ce529-b796-4213-9102-e615e8ec4ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:14.035232 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:33:14.126928 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 677/1351] 10.222.0.64 () {58 vars in 1018 bytes} [Mon May 23 15:33:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:18.343732 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9329dcd-6ca3-4f70-bbd4-6b0c0783cea7 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:18.344289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9329dcd-6ca3-4f70-bbd4-6b0c0783cea7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:18.344588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9329dcd-6ca3-4f70-bbd4-6b0c0783cea7 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:18.346459 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 675/1352] 10.222.0.64 () {54 vars in 921 bytes} [Mon May 23 15:33:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:18.351931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c07f1a-5083-4b0c-a11b-e02ce1dadfc3 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:18.352369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c07f1a-5083-4b0c-a11b-e02ce1dadfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:18.352609 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c07f1a-5083-4b0c-a11b-e02ce1dadfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:18.372015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-61c07f1a-5083-4b0c-a11b-e02ce1dadfc3 None None] MFA Rules not processed for user `04fbaeb7583a456d81cf79f6b101fb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:18.373245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-61c07f1a-5083-4b0c-a11b-e02ce1dadfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:18.381958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 678/1353] 10.222.0.64 () {58 vars in 1018 bytes} [Mon May 23 15:33:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:18.441026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=13 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:33:20.012700 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07251e7c-4250-4fc1-9c56-152ae9c641be None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:20.013362 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07251e7c-4250-4fc1-9c56-152ae9c641be None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:20.013362 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07251e7c-4250-4fc1-9c56-152ae9c641be None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:20.015000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 676/1354] 10.222.0.64 () {54 vars in 921 bytes} [Mon May 23 15:33:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:20.020194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7d8f9-0f0c-4683-a82a-ea96d9a31ba5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:20.020618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7d8f9-0f0c-4683-a82a-ea96d9a31ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:20.020776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7d8f9-0f0c-4683-a82a-ea96d9a31ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:20.039676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f4a7d8f9-0f0c-4683-a82a-ea96d9a31ba5 None None] MFA Rules not processed for user `04fbaeb7583a456d81cf79f6b101fb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:20.040868 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f4a7d8f9-0f0c-4683-a82a-ea96d9a31ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:20.050336 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 679/1355] 10.222.0.64 () {58 vars in 1018 bytes} [Mon May 23 15:33:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:23.981801 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5000d2-d2dd-4161-806d-720cfeef5ba2 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:23.982335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5000d2-d2dd-4161-806d-720cfeef5ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:23.982523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5000d2-d2dd-4161-806d-720cfeef5ba2 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:23.983967 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 677/1356] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:23.989349 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0df13-27d6-48ff-a991-00608c81b4f6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:23.989771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0df13-27d6-48ff-a991-00608c81b4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:23.989913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0df13-27d6-48ff-a991-00608c81b4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:24.008261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-63d0df13-27d6-48ff-a991-00608c81b4f6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:24.009735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-63d0df13-27d6-48ff-a991-00608c81b4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:24.018034 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 680/1357] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:24.077005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8ab54-a917-4714-aa12-e0560fd69e0d None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:24.077435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8ab54-a917-4714-aa12-e0560fd69e0d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:24.077639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8ab54-a917-4714-aa12-e0560fd69e0d None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:24.079880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 678/1358] 10.222.0.64 () {54 vars in 961 bytes} [Mon May 23 15:33:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:24.088884 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92257bf5-bf90-4ad1-82c7-961c0a77a35c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:24.090011 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92257bf5-bf90-4ad1-82c7-961c0a77a35c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:24.090585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92257bf5-bf90-4ad1-82c7-961c0a77a35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:24.163994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-92257bf5-bf90-4ad1-82c7-961c0a77a35c None None] MFA Rules not processed for user `8da96bed73794fee8f719ec904936164`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:24.165322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-92257bf5-bf90-4ad1-82c7-961c0a77a35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:24.262821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 681/1359] 10.222.0.64 () {58 vars in 1058 bytes} [Mon May 23 15:33:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:33:24.272044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8ab54-a917-4714-aa12-e0560fd69e0d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:24.288265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8ab54-a917-4714-aa12-e0560fd69e0d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:24.294689 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d5643-99fb-44e7-962f-ff7f5bc7e112 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:24.295582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d5643-99fb-44e7-962f-ff7f5bc7e112 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:24.295932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d5643-99fb-44e7-962f-ff7f5bc7e112 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:24.296280 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d5643-99fb-44e7-962f-ff7f5bc7e112 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:24.307296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d5643-99fb-44e7-962f-ff7f5bc7e112 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:24.309353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d5643-99fb-44e7-962f-ff7f5bc7e112 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:24.320248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 679/1360] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:26.786750 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224227a3-47d9-4f9c-b8e1-d16315805178 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:26.787318 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224227a3-47d9-4f9c-b8e1-d16315805178 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:26.787318 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224227a3-47d9-4f9c-b8e1-d16315805178 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:26.788982 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 682/1361] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:26.794288 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1333f4-73a7-4158-b91a-9d7454565722 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:26.795801 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1333f4-73a7-4158-b91a-9d7454565722 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:26.796173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1333f4-73a7-4158-b91a-9d7454565722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:26.816102 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6e1333f4-73a7-4158-b91a-9d7454565722 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:26.817337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6e1333f4-73a7-4158-b91a-9d7454565722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:26.827089 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 680/1362] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:26.872409 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224227a3-47d9-4f9c-b8e1-d16315805178 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:26.887681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224227a3-47d9-4f9c-b8e1-d16315805178 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:26.895476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b79d1d-3250-4dff-8d2d-70632446202d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:26.896156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b79d1d-3250-4dff-8d2d-70632446202d service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:26.896442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b79d1d-3250-4dff-8d2d-70632446202d service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:26.896723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b79d1d-3250-4dff-8d2d-70632446202d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:26.905605 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b79d1d-3250-4dff-8d2d-70632446202d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:26.907315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b79d1d-3250-4dff-8d2d-70632446202d service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:26.920890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 683/1363] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:29.661322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af10f6-d0c9-4806-9918-37366932612b None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:29.661876 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af10f6-d0c9-4806-9918-37366932612b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:29.661876 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af10f6-d0c9-4806-9918-37366932612b None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:29.663163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 681/1364] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:29.669149 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768dcb6-985d-4631-8a33-6f987c5c9e1c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:29.669590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768dcb6-985d-4631-8a33-6f987c5c9e1c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:29.669723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768dcb6-985d-4631-8a33-6f987c5c9e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:29.688557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f768dcb6-985d-4631-8a33-6f987c5c9e1c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:29.689793 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f768dcb6-985d-4631-8a33-6f987c5c9e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:29.698160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 684/1365] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:33:29.743811 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da749ffb-9241-4f7c-bdf2-d06b046865a3 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:29.744302 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da749ffb-9241-4f7c-bdf2-d06b046865a3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:29.744582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da749ffb-9241-4f7c-bdf2-d06b046865a3 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:29.746033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 682/1366] 10.222.0.64 () {54 vars in 961 bytes} [Mon May 23 15:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:29.752552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad85ef-da80-4ca7-8984-ce415ac931be None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:29.756847 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad85ef-da80-4ca7-8984-ce415ac931be None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:29.756847 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad85ef-da80-4ca7-8984-ce415ac931be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:29.771259 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fcad85ef-da80-4ca7-8984-ce415ac931be None None] MFA Rules not processed for user `8da96bed73794fee8f719ec904936164`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:29.772445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fcad85ef-da80-4ca7-8984-ce415ac931be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:29.782416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 685/1367] 10.222.0.64 () {58 vars in 1058 bytes} [Mon May 23 15:33:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:33:29.790396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da749ffb-9241-4f7c-bdf2-d06b046865a3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:29.804907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da749ffb-9241-4f7c-bdf2-d06b046865a3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:29.811223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666699b5-da52-4b51-98e6-fa6c1e5a452f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:29.811840 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666699b5-da52-4b51-98e6-fa6c1e5a452f service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:29.812122 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666699b5-da52-4b51-98e6-fa6c1e5a452f service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:29.812397 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666699b5-da52-4b51-98e6-fa6c1e5a452f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:29.820545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666699b5-da52-4b51-98e6-fa6c1e5a452f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:29.822443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666699b5-da52-4b51-98e6-fa6c1e5a452f service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:29.837191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 683/1368] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:32.303249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84999707-6ca9-49da-b224-75ddfa600792 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:32.303832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84999707-6ca9-49da-b224-75ddfa600792 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:32.303832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84999707-6ca9-49da-b224-75ddfa600792 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:32.305268 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 686/1369] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:33:32.312162 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d1b9b-1a3c-47ab-8e3b-acff3bbfd313 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:32.312773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d1b9b-1a3c-47ab-8e3b-acff3bbfd313 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:32.312773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d1b9b-1a3c-47ab-8e3b-acff3bbfd313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:32.332646 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-287d1b9b-1a3c-47ab-8e3b-acff3bbfd313 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:32.333957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f571170-7dea-4ed0-9e19-273b7a44249c None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:32.334175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-287d1b9b-1a3c-47ab-8e3b-acff3bbfd313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:32.334325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f571170-7dea-4ed0-9e19-273b7a44249c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:32.334513 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f571170-7dea-4ed0-9e19-273b7a44249c None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:32.336076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 687/1370] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:32.342709 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19543a-4d57-4f84-b3ef-f28f08c6aa64 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:32.343180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19543a-4d57-4f84-b3ef-f28f08c6aa64 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:32.343474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19543a-4d57-4f84-b3ef-f28f08c6aa64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:32.348101 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 684/1371] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:32.353265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:33:32.365820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ef19543a-4d57-4f84-b3ef-f28f08c6aa64 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:32.367039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ef19543a-4d57-4f84-b3ef-f28f08c6aa64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:32.375177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 688/1372] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:32.391292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d1b9b-1a3c-47ab-8e3b-acff3bbfd313 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:32.404489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d1b9b-1a3c-47ab-8e3b-acff3bbfd313 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:32.410230 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac5cce-f356-42c7-ab34-1bd802075d6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:32.410888 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac5cce-f356-42c7-ab34-1bd802075d6b service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:32.411176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac5cce-f356-42c7-ab34-1bd802075d6b service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:32.411456 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac5cce-f356-42c7-ab34-1bd802075d6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:32.420936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac5cce-f356-42c7-ab34-1bd802075d6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:32.422691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac5cce-f356-42c7-ab34-1bd802075d6b service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:32.434161 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 685/1373] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:32.691272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f7479-5c72-4269-b0e9-e74fec56fa7a None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:32.691436 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f7479-5c72-4269-b0e9-e74fec56fa7a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:32.694224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f7479-5c72-4269-b0e9-e74fec56fa7a None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:32.694224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 689/1374] 10.222.0.64 () {54 vars in 963 bytes} [Mon May 23 15:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:32.709109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5ea81-f656-4e05-a307-a523b86eabb6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:32.709109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5ea81-f656-4e05-a307-a523b86eabb6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:32.709109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5ea81-f656-4e05-a307-a523b86eabb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:32.769969 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-08a5ea81-f656-4e05-a307-a523b86eabb6 None None] MFA Rules not processed for user `8750b0450bf343f590c52b7d5c68d5df`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:32.771512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-08a5ea81-f656-4e05-a307-a523b86eabb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:32.842952 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 686/1375] 10.222.0.64 () {58 vars in 1060 bytes} [Mon May 23 15:33:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:32.850719 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f7479-5c72-4269-b0e9-e74fec56fa7a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:32.861875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f7479-5c72-4269-b0e9-e74fec56fa7a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:32.866935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb93791-e055-4a40-98f9-f6bb7b78ba38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:32.867735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb93791-e055-4a40-98f9-f6bb7b78ba38 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:32.868121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb93791-e055-4a40-98f9-f6bb7b78ba38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:32.868516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb93791-e055-4a40-98f9-f6bb7b78ba38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:32.879326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb93791-e055-4a40-98f9-f6bb7b78ba38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:32.880943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb93791-e055-4a40-98f9-f6bb7b78ba38 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:32.892778 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 690/1376] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:34.923353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a5159-f517-4603-832c-5747826a03c8 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:34.923858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a5159-f517-4603-832c-5747826a03c8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:34.923858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a5159-f517-4603-832c-5747826a03c8 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:34.925484 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 687/1377] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:34.931186 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff47d5-62a5-4a68-bf84-60b9197e960d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:34.931622 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff47d5-62a5-4a68-bf84-60b9197e960d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:34.931753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff47d5-62a5-4a68-bf84-60b9197e960d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:34.951297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2aff47d5-62a5-4a68-bf84-60b9197e960d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:34.952516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2aff47d5-62a5-4a68-bf84-60b9197e960d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:34.961724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 691/1378] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:35.006791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a5159-f517-4603-832c-5747826a03c8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:35.022046 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a5159-f517-4603-832c-5747826a03c8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:35.028377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c504aa7-b28c-4232-920b-67f3ce9d2f5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:35.029030 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c504aa7-b28c-4232-920b-67f3ce9d2f5c service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:35.029356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c504aa7-b28c-4232-920b-67f3ce9d2f5c service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:35.029636 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c504aa7-b28c-4232-920b-67f3ce9d2f5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:35.038348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c504aa7-b28c-4232-920b-67f3ce9d2f5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:35.039932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c504aa7-b28c-4232-920b-67f3ce9d2f5c service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:35.049924 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 688/1379] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:37.518057 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5ade7-a286-41c7-a1a1-49a1a4729bf5 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:37.518623 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5ade7-a286-41c7-a1a1-49a1a4729bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:37.518901 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5ade7-a286-41c7-a1a1-49a1a4729bf5 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:37.520359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 692/1380] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:37.525603 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debf13f-a982-4812-9e51-6ee7ed024078 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:37.526894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debf13f-a982-4812-9e51-6ee7ed024078 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:37.527339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debf13f-a982-4812-9e51-6ee7ed024078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:37.546207 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4debf13f-a982-4812-9e51-6ee7ed024078 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:37.547523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4debf13f-a982-4812-9e51-6ee7ed024078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:37.557100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 689/1381] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:37.599277 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5ade7-a286-41c7-a1a1-49a1a4729bf5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:37.614788 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5ade7-a286-41c7-a1a1-49a1a4729bf5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:37.621214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91214ba1-2bfd-407f-9e07-6a8ae81f48bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:37.621987 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91214ba1-2bfd-407f-9e07-6a8ae81f48bc service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:37.622649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91214ba1-2bfd-407f-9e07-6a8ae81f48bc service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:37.623002 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91214ba1-2bfd-407f-9e07-6a8ae81f48bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:37.631732 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91214ba1-2bfd-407f-9e07-6a8ae81f48bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:37.633461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91214ba1-2bfd-407f-9e07-6a8ae81f48bc service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:37.644623 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 693/1382] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:40.158542 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb40640-0873-4d14-9fe0-dab7cd49dff9 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:40.159369 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb40640-0873-4d14-9fe0-dab7cd49dff9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:40.159661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb40640-0873-4d14-9fe0-dab7cd49dff9 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:40.161033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 690/1383] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:40.167587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b5392-4a85-4d21-88b5-e5581e17f579 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:40.167587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b5392-4a85-4d21-88b5-e5581e17f579 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:40.168204 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b5392-4a85-4d21-88b5-e5581e17f579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:40.188533 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-669b5392-4a85-4d21-88b5-e5581e17f579 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:40.189995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-669b5392-4a85-4d21-88b5-e5581e17f579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:40.198685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 694/1384] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:40.241239 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb40640-0873-4d14-9fe0-dab7cd49dff9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:40.255800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb40640-0873-4d14-9fe0-dab7cd49dff9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:40.261765 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad77050-a8bd-4bdf-903e-3e30ad987751 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:40.262537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad77050-a8bd-4bdf-903e-3e30ad987751 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:40.263050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad77050-a8bd-4bdf-903e-3e30ad987751 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:40.263558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad77050-a8bd-4bdf-903e-3e30ad987751 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:40.272743 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad77050-a8bd-4bdf-903e-3e30ad987751 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:40.274581 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad77050-a8bd-4bdf-903e-3e30ad987751 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:40.286357 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 691/1385] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:42.653854 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d663c7d-a3a9-4710-89d1-aeb236607bf6 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:42.654473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d663c7d-a3a9-4710-89d1-aeb236607bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:42.654473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d663c7d-a3a9-4710-89d1-aeb236607bf6 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:42.655808 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 695/1386] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:42.661265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfcf5f-03de-4b35-81ac-c343b86831f4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:42.661830 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfcf5f-03de-4b35-81ac-c343b86831f4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:42.661978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfcf5f-03de-4b35-81ac-c343b86831f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:42.682458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-dcdfcf5f-03de-4b35-81ac-c343b86831f4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:42.683725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dcdfcf5f-03de-4b35-81ac-c343b86831f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:42.691693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 692/1387] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:42.737335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d663c7d-a3a9-4710-89d1-aeb236607bf6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:42.754498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d663c7d-a3a9-4710-89d1-aeb236607bf6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:42.760919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249777d-3a8d-45c8-b923-e44984a15f25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:42.761725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249777d-3a8d-45c8-b923-e44984a15f25 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:42.762080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249777d-3a8d-45c8-b923-e44984a15f25 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:42.762410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249777d-3a8d-45c8-b923-e44984a15f25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:42.772757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249777d-3a8d-45c8-b923-e44984a15f25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:42.774833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249777d-3a8d-45c8-b923-e44984a15f25 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:42.787866 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 696/1388] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:45.336045 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce194e41-5241-4b5e-8a3b-a633bb61794a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:45.336762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce194e41-5241-4b5e-8a3b-a633bb61794a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:45.336762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce194e41-5241-4b5e-8a3b-a633bb61794a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:45.338211 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 693/1389] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:45.343511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e1e64-c770-4a99-9723-20e410704514 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:45.343968 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e1e64-c770-4a99-9723-20e410704514 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:45.344072 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e1e64-c770-4a99-9723-20e410704514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:45.362172 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-011e1e64-c770-4a99-9723-20e410704514 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:45.363395 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-011e1e64-c770-4a99-9723-20e410704514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:45.371857 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 697/1390] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:45.418251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce194e41-5241-4b5e-8a3b-a633bb61794a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:45.433288 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce194e41-5241-4b5e-8a3b-a633bb61794a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:45.438859 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f63461-0033-4183-9fa2-86e83342716c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:45.439534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f63461-0033-4183-9fa2-86e83342716c service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:45.439828 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f63461-0033-4183-9fa2-86e83342716c service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:45.440111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f63461-0033-4183-9fa2-86e83342716c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:45.449448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f63461-0033-4183-9fa2-86e83342716c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:45.451113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f63461-0033-4183-9fa2-86e83342716c service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:45.461041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 694/1391] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:47.890550 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54264bcc-a38c-40d3-8fbe-64ebfaa8bb3c None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:47.891163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54264bcc-a38c-40d3-8fbe-64ebfaa8bb3c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:47.891163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54264bcc-a38c-40d3-8fbe-64ebfaa8bb3c None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:47.892473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 698/1392] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:47.897705 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb566e-3f08-45fc-b53f-4c0a5ab1eee2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:47.898185 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb566e-3f08-45fc-b53f-4c0a5ab1eee2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:47.898297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb566e-3f08-45fc-b53f-4c0a5ab1eee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:47.915590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8bdb566e-3f08-45fc-b53f-4c0a5ab1eee2 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:47.916798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8bdb566e-3f08-45fc-b53f-4c0a5ab1eee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:47.924880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 695/1393] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:47.969302 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54264bcc-a38c-40d3-8fbe-64ebfaa8bb3c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:47.983745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54264bcc-a38c-40d3-8fbe-64ebfaa8bb3c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:47.989151 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941e16c-143d-43f7-9810-4f7d6cc7718f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:47.989798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941e16c-143d-43f7-9810-4f7d6cc7718f service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:47.990087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941e16c-143d-43f7-9810-4f7d6cc7718f service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:47.990376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941e16c-143d-43f7-9810-4f7d6cc7718f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:47.998924 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941e16c-143d-43f7-9810-4f7d6cc7718f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:48.000527 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941e16c-143d-43f7-9810-4f7d6cc7718f service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:48.010388 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 699/1394] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:48.952855 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75908307-d255-45f7-9f49-b34823d4c026 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:48.953427 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75908307-d255-45f7-9f49-b34823d4c026 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:48.953427 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75908307-d255-45f7-9f49-b34823d4c026 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:48.954759 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 696/1395] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:48.960200 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a052c3a-cf2c-47d4-8e8e-c7c6b783808d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:48.960627 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a052c3a-cf2c-47d4-8e8e-c7c6b783808d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:48.960774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a052c3a-cf2c-47d4-8e8e-c7c6b783808d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:48.980100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8a052c3a-cf2c-47d4-8e8e-c7c6b783808d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:48.981329 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8a052c3a-cf2c-47d4-8e8e-c7c6b783808d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:48.989016 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 700/1396] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:49.043966 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e7a78-a13e-45eb-b816-c20c98cc411f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:49.044259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e7a78-a13e-45eb-b816-c20c98cc411f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:49.044540 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e7a78-a13e-45eb-b816-c20c98cc411f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:49.045958 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 697/1397] 10.222.0.64 () {54 vars in 962 bytes} [Mon May 23 15:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:49.050857 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26c35a-cb14-43fb-937f-6a95c17682c0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:49.051414 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26c35a-cb14-43fb-937f-6a95c17682c0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:49.051693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26c35a-cb14-43fb-937f-6a95c17682c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:49.100432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ea26c35a-cb14-43fb-937f-6a95c17682c0 None None] MFA Rules not processed for user `45ee7a6c342548f98e7dc73f4f71a66d`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:49.101678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ea26c35a-cb14-43fb-937f-6a95c17682c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:49.169350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 701/1398] 10.222.0.64 () {58 vars in 1059 bytes} [Mon May 23 15:33:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:49.174919 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e7a78-a13e-45eb-b816-c20c98cc411f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:49.180493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:33:49.185836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e7a78-a13e-45eb-b816-c20c98cc411f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:49.189375 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7045ca-37ea-4512-a1f6-b5a4b680e7ba service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee7a6c342548f98e7dc73f4f71a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:49.191175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7045ca-37ea-4512-a1f6-b5a4b680e7ba service glance] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:49.191175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7045ca-37ea-4512-a1f6-b5a4b680e7ba service glance] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:49.191175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7045ca-37ea-4512-a1f6-b5a4b680e7ba service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:49.200385 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7045ca-37ea-4512-a1f6-b5a4b680e7ba service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:49.201889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7045ca-37ea-4512-a1f6-b5a4b680e7ba service glance] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:49.211342 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 698/1399] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:49.909120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600899c-51f6-4487-b44c-2f381734c2b7 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:49.909436 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600899c-51f6-4487-b44c-2f381734c2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:49.909718 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600899c-51f6-4487-b44c-2f381734c2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:49.959006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c600899c-51f6-4487-b44c-2f381734c2b7 None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:49.960215 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c600899c-51f6-4487-b44c-2f381734c2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:49.986440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:33:50.024638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 702/1400] 10.222.0.64 () {58 vars in 1009 bytes} [Mon May 23 15:33:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:50.066527 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdff09-d1c5-4912-9744-855423dfbdf1 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:50.067008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdff09-d1c5-4912-9744-855423dfbdf1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:50.067099 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdff09-d1c5-4912-9744-855423dfbdf1 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:50.068440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 699/1401] 10.222.0.64 () {54 vars in 942 bytes} [Mon May 23 15:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:50.075336 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34299b96-fcce-420a-b31f-0fd238adddd8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:50.075712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34299b96-fcce-420a-b31f-0fd238adddd8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:50.076062 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34299b96-fcce-420a-b31f-0fd238adddd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:50.127113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-34299b96-fcce-420a-b31f-0fd238adddd8 None None] MFA Rules not processed for user `f8c79c36c7204368816593f0e473191b`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:50.128308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-34299b96-fcce-420a-b31f-0fd238adddd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:50.202581 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 703/1402] 10.222.0.64 () {58 vars in 1039 bytes} [Mon May 23 15:33:50 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:50.211699 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdff09-d1c5-4912-9744-855423dfbdf1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:50.225643 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdff09-d1c5-4912-9744-855423dfbdf1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:50.231784 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02ff13-7458-4e16-a74a-2729887fbb97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:50.232634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02ff13-7458-4e16-a74a-2729887fbb97 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:50.232936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02ff13-7458-4e16-a74a-2729887fbb97 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:50.233326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02ff13-7458-4e16-a74a-2729887fbb97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:50.243150 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02ff13-7458-4e16-a74a-2729887fbb97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:50.245445 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02ff13-7458-4e16-a74a-2729887fbb97 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:50.254897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 700/1403] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:33:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:33:50.433763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcdf6c-9c22-40a0-8816-208fc2510b10 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:50.434089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcdf6c-9c22-40a0-8816-208fc2510b10 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:50.434369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcdf6c-9c22-40a0-8816-208fc2510b10 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:50.435748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 704/1404] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:50.441110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becd672-610d-481d-9c86-5d4fe41d15d6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:50.441502 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becd672-610d-481d-9c86-5d4fe41d15d6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:50.441792 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becd672-610d-481d-9c86-5d4fe41d15d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:50.459796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5becd672-610d-481d-9c86-5d4fe41d15d6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:50.460932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5becd672-610d-481d-9c86-5d4fe41d15d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:50.468958 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 701/1405] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:33:50.514221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcdf6c-9c22-40a0-8816-208fc2510b10 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:50.528739 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcdf6c-9c22-40a0-8816-208fc2510b10 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:50.534225 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f602a7-0131-4ee5-9231-f1cf19d9c795 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:50.534874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f602a7-0131-4ee5-9231-f1cf19d9c795 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:50.535165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f602a7-0131-4ee5-9231-f1cf19d9c795 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:50.535446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f602a7-0131-4ee5-9231-f1cf19d9c795 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:50.544592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f602a7-0131-4ee5-9231-f1cf19d9c795 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:50.546373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f602a7-0131-4ee5-9231-f1cf19d9c795 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:50.557205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 705/1406] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:33:52.990087 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfe568-1a49-4516-8d16-b7aa2cedbaec None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:52.990619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfe568-1a49-4516-8d16-b7aa2cedbaec None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:52.990619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfe568-1a49-4516-8d16-b7aa2cedbaec None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:52.991984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 702/1407] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:33:52.998097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cd086-2579-4c4e-bc8b-90300943b4d2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:52.998515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cd086-2579-4c4e-bc8b-90300943b4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:52.998689 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cd086-2579-4c4e-bc8b-90300943b4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:53.018009 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-aa3cd086-2579-4c4e-bc8b-90300943b4d2 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:33:53.019520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-aa3cd086-2579-4c4e-bc8b-90300943b4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:33:53.028560 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 706/1408] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:33:53.080973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfe568-1a49-4516-8d16-b7aa2cedbaec None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:33:53.095934 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfe568-1a49-4516-8d16-b7aa2cedbaec None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:33:53.104214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913a755-984e-45b9-a6c1-b818e4a5a06b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:33:53.104870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913a755-984e-45b9-a6c1-b818e4a5a06b service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:33:53.105190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913a755-984e-45b9-a6c1-b818e4a5a06b service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:33:53.105472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913a755-984e-45b9-a6c1-b818e4a5a06b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:33:53.115200 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913a755-984e-45b9-a6c1-b818e4a5a06b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:33:53.117109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913a755-984e-45b9-a6c1-b818e4a5a06b service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:33:53.127097 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 703/1409] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:33:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:03.124510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=68 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:34:03.124973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=20 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:34:14.828476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c10c6f-f7b3-4cb7-8acb-cafdde1f986b None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:14.828925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c10c6f-f7b3-4cb7-8acb-cafdde1f986b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:14.829047 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c10c6f-f7b3-4cb7-8acb-cafdde1f986b None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:14.830470 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 707/1410] 10.222.0.64 () {54 vars in 918 bytes} [Mon May 23 15:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:14.835754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b1e8a-b03b-450d-a8d2-21a34d30907f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:14.836197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b1e8a-b03b-450d-a8d2-21a34d30907f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:14.836307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b1e8a-b03b-450d-a8d2-21a34d30907f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:14.855021 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-607b1e8a-b03b-450d-a8d2-21a34d30907f None None] MFA Rules not processed for user `04fbaeb7583a456d81cf79f6b101fb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:14.856380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-607b1e8a-b03b-450d-a8d2-21a34d30907f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:14.864735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 704/1411] 10.222.0.64 () {58 vars in 1015 bytes} [Mon May 23 15:34:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:14.878747 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b1363-8abd-472e-bb14-37f9e74e1577 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:14.879039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b1363-8abd-472e-bb14-37f9e74e1577 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:14.879321 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b1363-8abd-472e-bb14-37f9e74e1577 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:14.880744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 708/1412] 10.222.0.64 () {54 vars in 918 bytes} [Mon May 23 15:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:14.885279 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1999b-682d-4ca5-8538-a28635935888 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:14.885570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1999b-682d-4ca5-8538-a28635935888 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:14.885852 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1999b-682d-4ca5-8538-a28635935888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:14.903496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-00e1999b-682d-4ca5-8538-a28635935888 None None] MFA Rules not processed for user `8da96bed73794fee8f719ec904936164`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:14.904669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-00e1999b-682d-4ca5-8538-a28635935888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:14.913517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 705/1413] 10.222.0.64 () {58 vars in 1015 bytes} [Mon May 23 15:34:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:17.566334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdce3f-e701-49ad-82a4-a04f25dd3763 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:17.566909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdce3f-e701-49ad-82a4-a04f25dd3763 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:17.566909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdce3f-e701-49ad-82a4-a04f25dd3763 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:17.568337 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:17.574274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06033219-39d4-4a30-a468-4d3cffd8692e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:17.574695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06033219-39d4-4a30-a468-4d3cffd8692e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:17.574841 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06033219-39d4-4a30-a468-4d3cffd8692e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:17.593287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-06033219-39d4-4a30-a468-4d3cffd8692e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:17.594492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-06033219-39d4-4a30-a468-4d3cffd8692e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:17.602350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:17.644702 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdce3f-e701-49ad-82a4-a04f25dd3763 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:17.659648 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdce3f-e701-49ad-82a4-a04f25dd3763 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:17.665256 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952013f3-f25e-41b5-8ca4-b4ea2ab43cb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:17.665893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952013f3-f25e-41b5-8ca4-b4ea2ab43cb4 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:17.666179 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952013f3-f25e-41b5-8ca4-b4ea2ab43cb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:17.666454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952013f3-f25e-41b5-8ca4-b4ea2ab43cb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:17.674784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952013f3-f25e-41b5-8ca4-b4ea2ab43cb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:17.676927 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952013f3-f25e-41b5-8ca4-b4ea2ab43cb4 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:17.686848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 710/1416] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:18.661035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b516a2-9e54-4852-95cb-0b609314bf62 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:18.661661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b516a2-9e54-4852-95cb-0b609314bf62 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:18.661661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b516a2-9e54-4852-95cb-0b609314bf62 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:18.663005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:18.668289 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2614ab5-35aa-487c-be5f-683182f33cd1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:18.668700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2614ab5-35aa-487c-be5f-683182f33cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:18.668863 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2614ab5-35aa-487c-be5f-683182f33cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:18.687799 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c2614ab5-35aa-487c-be5f-683182f33cd1 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:18.689007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c2614ab5-35aa-487c-be5f-683182f33cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:18.697612 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:34:18.861684 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b516a2-9e54-4852-95cb-0b609314bf62 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:18.876577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b516a2-9e54-4852-95cb-0b609314bf62 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:18.882050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a6092-b865-40f8-8eb6-85e1dc8be64e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:18.882724 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a6092-b865-40f8-8eb6-85e1dc8be64e service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:18.883010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a6092-b865-40f8-8eb6-85e1dc8be64e service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:18.883357 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a6092-b865-40f8-8eb6-85e1dc8be64e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:18.891477 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a6092-b865-40f8-8eb6-85e1dc8be64e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:18.893042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a6092-b865-40f8-8eb6-85e1dc8be64e service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:18.902229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 708/1419] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:19.833519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c57b6b-ac27-433e-a778-44507defd904 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:19.834066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c57b6b-ac27-433e-a778-44507defd904 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:19.834202 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c57b6b-ac27-433e-a778-44507defd904 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:19.835527 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:19.841121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106a103-be55-4ca3-803e-b032b51cb14f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:19.841580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106a103-be55-4ca3-803e-b032b51cb14f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:19.841691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106a103-be55-4ca3-803e-b032b51cb14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:19.859434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c106a103-be55-4ca3-803e-b032b51cb14f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:19.860671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c106a103-be55-4ca3-803e-b032b51cb14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:19.868536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:20.019826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c57b6b-ac27-433e-a778-44507defd904 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:20.034503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c57b6b-ac27-433e-a778-44507defd904 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:20.039915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f6e6f-c602-4f91-bac7-5a56f5545982 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:20.040600 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f6e6f-c602-4f91-bac7-5a56f5545982 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:20.040887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f6e6f-c602-4f91-bac7-5a56f5545982 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:20.041205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f6e6f-c602-4f91-bac7-5a56f5545982 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:20.049235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f6e6f-c602-4f91-bac7-5a56f5545982 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:20.051094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f6e6f-c602-4f91-bac7-5a56f5545982 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:20.061261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 713/1422] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:20.970092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1eb0f-fb87-4b3c-86f5-1080fd7d73ef None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:20.970612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1eb0f-fb87-4b3c-86f5-1080fd7d73ef None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:20.970612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1eb0f-fb87-4b3c-86f5-1080fd7d73ef None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:20.971951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:20.977494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866da4f2-e948-43cf-bf12-e49b222622cd None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:20.977891 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866da4f2-e948-43cf-bf12-e49b222622cd None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:20.978067 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866da4f2-e948-43cf-bf12-e49b222622cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:20.996986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-866da4f2-e948-43cf-bf12-e49b222622cd None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:20.998261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-866da4f2-e948-43cf-bf12-e49b222622cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:21.006026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:21.156143 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1eb0f-fb87-4b3c-86f5-1080fd7d73ef None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:21.171282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1eb0f-fb87-4b3c-86f5-1080fd7d73ef None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:21.177217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d36b8-7a8c-48a6-9b07-8692aae24f98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:21.177887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d36b8-7a8c-48a6-9b07-8692aae24f98 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:21.178171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d36b8-7a8c-48a6-9b07-8692aae24f98 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:21.178447 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d36b8-7a8c-48a6-9b07-8692aae24f98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:21.187254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d36b8-7a8c-48a6-9b07-8692aae24f98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:21.188851 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d36b8-7a8c-48a6-9b07-8692aae24f98 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:21.198344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 711/1425] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:22.107036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0feb66-b709-4159-94a0-551226ffb9a5 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:22.107605 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0feb66-b709-4159-94a0-551226ffb9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:22.107707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0feb66-b709-4159-94a0-551226ffb9a5 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:22.109210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:22.114297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875e941-161b-479f-8c35-ff77048b4922 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:22.114741 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875e941-161b-479f-8c35-ff77048b4922 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:22.114884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875e941-161b-479f-8c35-ff77048b4922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:22.134212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a875e941-161b-479f-8c35-ff77048b4922 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:22.135419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a875e941-161b-479f-8c35-ff77048b4922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:22.143321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:22.294559 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0feb66-b709-4159-94a0-551226ffb9a5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:22.309560 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0feb66-b709-4159-94a0-551226ffb9a5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:22.314882 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1353dd-c304-4d79-980a-12c29a70374e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:22.315544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1353dd-c304-4d79-980a-12c29a70374e service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:22.315829 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1353dd-c304-4d79-980a-12c29a70374e service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:22.316104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1353dd-c304-4d79-980a-12c29a70374e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:22.324451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1353dd-c304-4d79-980a-12c29a70374e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:22.326077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1353dd-c304-4d79-980a-12c29a70374e service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:22.335207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 716/1428] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:23.246341 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1914935-cc31-4e69-85af-cb583ec2e595 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:23.246913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1914935-cc31-4e69-85af-cb583ec2e595 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:23.246913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1914935-cc31-4e69-85af-cb583ec2e595 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:23.248522 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:23.253515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288e943-7018-40ce-9d78-e0aebaf041c8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:23.253947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288e943-7018-40ce-9d78-e0aebaf041c8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:23.254094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288e943-7018-40ce-9d78-e0aebaf041c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:23.272255 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c288e943-7018-40ce-9d78-e0aebaf041c8 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:23.273722 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c288e943-7018-40ce-9d78-e0aebaf041c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:23.281595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:23.433754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1914935-cc31-4e69-85af-cb583ec2e595 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:23.448242 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1914935-cc31-4e69-85af-cb583ec2e595 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:23.453570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17deaf11-98a1-48ec-9010-cf8e89c11021 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:23.454208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17deaf11-98a1-48ec-9010-cf8e89c11021 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:23.454494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17deaf11-98a1-48ec-9010-cf8e89c11021 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:23.454778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17deaf11-98a1-48ec-9010-cf8e89c11021 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:23.463495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17deaf11-98a1-48ec-9010-cf8e89c11021 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:23.465076 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17deaf11-98a1-48ec-9010-cf8e89c11021 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:23.474382 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 714/1431] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:24.394617 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352eaf9-a67a-4031-ba70-0b47a6620124 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:24.395144 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352eaf9-a67a-4031-ba70-0b47a6620124 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:24.395144 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352eaf9-a67a-4031-ba70-0b47a6620124 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:24.396484 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:24.402234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceca48f-ad2c-4668-871d-b571f328ad04 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:24.402680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceca48f-ad2c-4668-871d-b571f328ad04 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:24.402791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceca48f-ad2c-4668-871d-b571f328ad04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:24.420533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2ceca48f-ad2c-4668-871d-b571f328ad04 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:24.421749 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2ceca48f-ad2c-4668-871d-b571f328ad04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:24.429688 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:24.579937 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352eaf9-a67a-4031-ba70-0b47a6620124 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:24.594541 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352eaf9-a67a-4031-ba70-0b47a6620124 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:24.599783 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c1082-03be-409d-8aa1-267af5b137a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:24.600412 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c1082-03be-409d-8aa1-267af5b137a6 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:24.600714 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c1082-03be-409d-8aa1-267af5b137a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:24.601045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c1082-03be-409d-8aa1-267af5b137a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:24.609685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c1082-03be-409d-8aa1-267af5b137a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:24.611297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c1082-03be-409d-8aa1-267af5b137a6 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:24.620699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 719/1434] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:25.543741 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9811739-2461-425b-b931-bb49d79a1fbf None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:25.544207 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9811739-2461-425b-b931-bb49d79a1fbf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:25.544682 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9811739-2461-425b-b931-bb49d79a1fbf None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:25.547204 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:34:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:25.555291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637b28c-e3d7-44a6-90f2-6685c64d4fb5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:25.561023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637b28c-e3d7-44a6-90f2-6685c64d4fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:25.561023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637b28c-e3d7-44a6-90f2-6685c64d4fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:25.574373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e637b28c-e3d7-44a6-90f2-6685c64d4fb5 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:25.575580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e637b28c-e3d7-44a6-90f2-6685c64d4fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:25.583405 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:25.736029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9811739-2461-425b-b931-bb49d79a1fbf None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:25.750982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9811739-2461-425b-b931-bb49d79a1fbf None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:25.756337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b33352-dbfc-4963-9d81-ab80a8b5c6e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:25.756984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b33352-dbfc-4963-9d81-ab80a8b5c6e7 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:25.757309 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b33352-dbfc-4963-9d81-ab80a8b5c6e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:25.757595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b33352-dbfc-4963-9d81-ab80a8b5c6e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:25.766477 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b33352-dbfc-4963-9d81-ab80a8b5c6e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:25.768078 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b33352-dbfc-4963-9d81-ab80a8b5c6e7 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:25.778257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 717/1437] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:26.747472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287c129-ad65-4d3f-85ec-d5c11e2ce0d8 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:26.747944 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287c129-ad65-4d3f-85ec-d5c11e2ce0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:26.748041 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287c129-ad65-4d3f-85ec-d5c11e2ce0d8 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:26.749444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:26.754312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e41e9f-2a73-474d-8e43-4b6f9b6cee8b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:26.754741 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e41e9f-2a73-474d-8e43-4b6f9b6cee8b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:26.754876 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e41e9f-2a73-474d-8e43-4b6f9b6cee8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:26.772984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-90e41e9f-2a73-474d-8e43-4b6f9b6cee8b None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:26.774224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-90e41e9f-2a73-474d-8e43-4b6f9b6cee8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:26.781954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:26.930102 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287c129-ad65-4d3f-85ec-d5c11e2ce0d8 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:26.944292 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287c129-ad65-4d3f-85ec-d5c11e2ce0d8 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:26.950517 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f377ba-117b-4abd-b23b-c0329774a7cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:26.951099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f377ba-117b-4abd-b23b-c0329774a7cf service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:26.951381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f377ba-117b-4abd-b23b-c0329774a7cf service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:26.951661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f377ba-117b-4abd-b23b-c0329774a7cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:26.959838 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f377ba-117b-4abd-b23b-c0329774a7cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:26.961639 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f377ba-117b-4abd-b23b-c0329774a7cf service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:26.970806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 722/1440] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:34:29.046340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf207299-d7c6-41db-ad6d-9786ccac7ebc None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:29.046922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf207299-d7c6-41db-ad6d-9786ccac7ebc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:29.046922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf207299-d7c6-41db-ad6d-9786ccac7ebc None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:29.048264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:29.053832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5967b83-1f56-45bb-953e-321499518a6d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:29.054268 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5967b83-1f56-45bb-953e-321499518a6d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:29.054411 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5967b83-1f56-45bb-953e-321499518a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:29.076420 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b5967b83-1f56-45bb-953e-321499518a6d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:29.078551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b5967b83-1f56-45bb-953e-321499518a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:29.086467 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:29.125945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf207299-d7c6-41db-ad6d-9786ccac7ebc None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:29.137322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf207299-d7c6-41db-ad6d-9786ccac7ebc None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:29.140848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab1614-f16f-42f5-a10f-79326fb9fb12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:29.141510 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab1614-f16f-42f5-a10f-79326fb9fb12 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:29.141797 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab1614-f16f-42f5-a10f-79326fb9fb12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:29.142718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab1614-f16f-42f5-a10f-79326fb9fb12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:29.151427 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab1614-f16f-42f5-a10f-79326fb9fb12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:29.152850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab1614-f16f-42f5-a10f-79326fb9fb12 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:29.162329 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 720/1443] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:31.506103 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c742fd9-6633-42b1-b01e-dd4f176701e0 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:31.506693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c742fd9-6633-42b1-b01e-dd4f176701e0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:31.506693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c742fd9-6633-42b1-b01e-dd4f176701e0 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:31.508240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:31.513902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131651d5-afe4-4673-8a34-30bcc1ad39fa None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:31.514369 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131651d5-afe4-4673-8a34-30bcc1ad39fa None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:31.514481 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131651d5-afe4-4673-8a34-30bcc1ad39fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:31.532845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-131651d5-afe4-4673-8a34-30bcc1ad39fa None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:31.534062 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-131651d5-afe4-4673-8a34-30bcc1ad39fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:31.542232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:31.579495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c742fd9-6633-42b1-b01e-dd4f176701e0 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:31.590962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c742fd9-6633-42b1-b01e-dd4f176701e0 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:31.594434 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96a487-7174-49e9-a815-599844511fb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:31.595074 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96a487-7174-49e9-a815-599844511fb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:31.595346 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96a487-7174-49e9-a815-599844511fb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:31.595620 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96a487-7174-49e9-a815-599844511fb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:31.605390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96a487-7174-49e9-a815-599844511fb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:31.606868 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96a487-7174-49e9-a815-599844511fb7 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:31.615945 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 725/1446] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:34.206487 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed089a-90ad-409a-adbc-9f1f8d86ef40 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:34.207176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed089a-90ad-409a-adbc-9f1f8d86ef40 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:34.207176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed089a-90ad-409a-adbc-9f1f8d86ef40 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:34.208495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:34.214743 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6674444-b51c-41f5-ad31-c449affc8515 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:34.215187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6674444-b51c-41f5-ad31-c449affc8515 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:34.215291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6674444-b51c-41f5-ad31-c449affc8515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:34.233667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e6674444-b51c-41f5-ad31-c449affc8515 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:34.234866 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e6674444-b51c-41f5-ad31-c449affc8515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:34.243096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:34.281130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed089a-90ad-409a-adbc-9f1f8d86ef40 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:34.291784 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed089a-90ad-409a-adbc-9f1f8d86ef40 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:34.295236 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8f2e1-8575-41f2-98ed-9a171969d097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:34.295889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8f2e1-8575-41f2-98ed-9a171969d097 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:34.296178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8f2e1-8575-41f2-98ed-9a171969d097 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:34.296478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8f2e1-8575-41f2-98ed-9a171969d097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:34.306009 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8f2e1-8575-41f2-98ed-9a171969d097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:34.307423 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8f2e1-8575-41f2-98ed-9a171969d097 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:34.316660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 723/1449] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:36.637636 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304a654-c07d-447d-9653-c5d21bfe5762 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:36.638216 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304a654-c07d-447d-9653-c5d21bfe5762 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:36.638216 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304a654-c07d-447d-9653-c5d21bfe5762 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:36.639545 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:36.645010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98008b8-f001-483a-9440-4b15421c3eba None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:36.645468 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98008b8-f001-483a-9440-4b15421c3eba None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:36.645619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98008b8-f001-483a-9440-4b15421c3eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:36.664178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d98008b8-f001-483a-9440-4b15421c3eba None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:36.665412 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d98008b8-f001-483a-9440-4b15421c3eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:36.673251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:34:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:36.712049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304a654-c07d-447d-9653-c5d21bfe5762 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:36.722664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304a654-c07d-447d-9653-c5d21bfe5762 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:36.727007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c74d3-e1b2-4731-b074-eea4ff56a8e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:36.727654 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c74d3-e1b2-4731-b074-eea4ff56a8e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:36.727940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c74d3-e1b2-4731-b074-eea4ff56a8e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:36.728248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c74d3-e1b2-4731-b074-eea4ff56a8e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:36.737176 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c74d3-e1b2-4731-b074-eea4ff56a8e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:36.738613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c74d3-e1b2-4731-b074-eea4ff56a8e0 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:36.748012 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 728/1452] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:39.001022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50054f38-62ff-458c-b40f-2f2e6115a206 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:39.001591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50054f38-62ff-458c-b40f-2f2e6115a206 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:39.001591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50054f38-62ff-458c-b40f-2f2e6115a206 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:39.002962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:39.008218 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8667d0-2bc9-4fa0-b09b-3aeeebe35872 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:39.008661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8667d0-2bc9-4fa0-b09b-3aeeebe35872 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:39.008774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8667d0-2bc9-4fa0-b09b-3aeeebe35872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:39.027306 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-bc8667d0-2bc9-4fa0-b09b-3aeeebe35872 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:39.028500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bc8667d0-2bc9-4fa0-b09b-3aeeebe35872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:39.036168 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:34:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:39.073905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50054f38-62ff-458c-b40f-2f2e6115a206 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:39.084519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50054f38-62ff-458c-b40f-2f2e6115a206 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:39.088599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f005b464-fe98-4aae-8570-11b459f466cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:39.095011 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005b464-fe98-4aae-8570-11b459f466cd service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:39.095011 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005b464-fe98-4aae-8570-11b459f466cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:39.095011 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005b464-fe98-4aae-8570-11b459f466cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:39.098497 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f005b464-fe98-4aae-8570-11b459f466cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:39.099917 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f005b464-fe98-4aae-8570-11b459f466cd service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:39.110099 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 726/1455] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:41.345572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99009-a894-4b38-9185-a5c48799e11e None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:41.346173 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99009-a894-4b38-9185-a5c48799e11e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:41.346173 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99009-a894-4b38-9185-a5c48799e11e None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:41.347489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:41.353626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16062fc-cab6-4a29-bee1-b8c5462b951e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:41.354060 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16062fc-cab6-4a29-bee1-b8c5462b951e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:41.354199 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16062fc-cab6-4a29-bee1-b8c5462b951e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:41.372470 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d16062fc-cab6-4a29-bee1-b8c5462b951e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:41.373692 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d16062fc-cab6-4a29-bee1-b8c5462b951e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:41.381318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:34:41.419110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99009-a894-4b38-9185-a5c48799e11e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:41.429836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99009-a894-4b38-9185-a5c48799e11e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:41.433384 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b54a82-10af-48ef-aa67-ad52e1c87959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:41.434013 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b54a82-10af-48ef-aa67-ad52e1c87959 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:41.434295 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b54a82-10af-48ef-aa67-ad52e1c87959 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:41.434569 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b54a82-10af-48ef-aa67-ad52e1c87959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:41.443324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b54a82-10af-48ef-aa67-ad52e1c87959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:41.445397 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b54a82-10af-48ef-aa67-ad52e1c87959 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:41.454958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 731/1458] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:43.704819 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1940a-9d7b-40f9-a794-20e8b2f0505b None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:43.705315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1940a-9d7b-40f9-a794-20e8b2f0505b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:43.705434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1940a-9d7b-40f9-a794-20e8b2f0505b None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:43.706800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:43.713535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346a068-be78-468c-8370-ca9e220c62e4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:43.713996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346a068-be78-468c-8370-ca9e220c62e4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:43.714094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346a068-be78-468c-8370-ca9e220c62e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:43.723864 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=70 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:34:43.732723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e346a068-be78-468c-8370-ca9e220c62e4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:43.735426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e346a068-be78-468c-8370-ca9e220c62e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:43.741976 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:34:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:43.779407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1940a-9d7b-40f9-a794-20e8b2f0505b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:43.784789 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=73 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:34:43.790353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1940a-9d7b-40f9-a794-20e8b2f0505b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:43.793841 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab007e6-f324-4702-9cc2-5b01356e8d93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:43.794471 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab007e6-f324-4702-9cc2-5b01356e8d93 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:43.794751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab007e6-f324-4702-9cc2-5b01356e8d93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:43.795026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab007e6-f324-4702-9cc2-5b01356e8d93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:43.803973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab007e6-f324-4702-9cc2-5b01356e8d93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:43.805620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab007e6-f324-4702-9cc2-5b01356e8d93 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:43.815641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 729/1461] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:46.077303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8baee62-7019-466f-95c1-56539c9c1853 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:46.077849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8baee62-7019-466f-95c1-56539c9c1853 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:46.077849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8baee62-7019-466f-95c1-56539c9c1853 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:46.079271 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:46.084623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ca83f-bf42-4ba0-bb6f-185f9c14529a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:46.085838 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ca83f-bf42-4ba0-bb6f-185f9c14529a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:46.085963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ca83f-bf42-4ba0-bb6f-185f9c14529a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:46.103524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-079ca83f-bf42-4ba0-bb6f-185f9c14529a None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:46.104712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-079ca83f-bf42-4ba0-bb6f-185f9c14529a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:46.112522 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:34:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:46.149663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8baee62-7019-466f-95c1-56539c9c1853 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:46.160084 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8baee62-7019-466f-95c1-56539c9c1853 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:46.163721 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a08045-4aca-475c-b72f-f924f372fc1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:46.164349 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a08045-4aca-475c-b72f-f924f372fc1a service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:46.164632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a08045-4aca-475c-b72f-f924f372fc1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:46.164910 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a08045-4aca-475c-b72f-f924f372fc1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:46.173566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a08045-4aca-475c-b72f-f924f372fc1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:46.174986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a08045-4aca-475c-b72f-f924f372fc1a service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:46.184498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 734/1464] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:48.398732 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c737cec-d245-4c19-b578-93e5de814f73 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:48.399339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c737cec-d245-4c19-b578-93e5de814f73 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:48.399339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c737cec-d245-4c19-b578-93e5de814f73 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:48.400705 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:34:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:48.406534 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cad0a3-a701-4dac-9fb7-f750c15b5c9e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:48.407002 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cad0a3-a701-4dac-9fb7-f750c15b5c9e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:48.407100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cad0a3-a701-4dac-9fb7-f750c15b5c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:48.425363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-46cad0a3-a701-4dac-9fb7-f750c15b5c9e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:48.427443 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-46cad0a3-a701-4dac-9fb7-f750c15b5c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:48.439268 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:34:48.476041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c737cec-d245-4c19-b578-93e5de814f73 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:48.487039 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c737cec-d245-4c19-b578-93e5de814f73 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:48.490564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec301273-8b4d-47c8-8fb0-489f8ccc6e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:48.491780 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec301273-8b4d-47c8-8fb0-489f8ccc6e65 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:48.492068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec301273-8b4d-47c8-8fb0-489f8ccc6e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:48.492344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec301273-8b4d-47c8-8fb0-489f8ccc6e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:48.501379 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec301273-8b4d-47c8-8fb0-489f8ccc6e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:48.502838 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec301273-8b4d-47c8-8fb0-489f8ccc6e65 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:48.512350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 732/1467] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:34:50.699240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c149a8-5159-4bab-90a3-8298731e8787 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:50.699836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c149a8-5159-4bab-90a3-8298731e8787 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:50.699836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c149a8-5159-4bab-90a3-8298731e8787 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:50.701146 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:34:50.706544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fba5e-fc93-4302-9c2c-561a519ba5c9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:50.706960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fba5e-fc93-4302-9c2c-561a519ba5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:50.707114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fba5e-fc93-4302-9c2c-561a519ba5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:50.725632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e94fba5e-fc93-4302-9c2c-561a519ba5c9 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:34:50.726950 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e94fba5e-fc93-4302-9c2c-561a519ba5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:34:50.734619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:34:50.771199 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c149a8-5159-4bab-90a3-8298731e8787 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:34:50.781971 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c149a8-5159-4bab-90a3-8298731e8787 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:34:50.785433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f6eab-daaa-4696-961e-5296658a126c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:34:50.786071 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f6eab-daaa-4696-961e-5296658a126c service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:34:50.786351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f6eab-daaa-4696-961e-5296658a126c service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:34:50.786627 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f6eab-daaa-4696-961e-5296658a126c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:34:50.795691 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f6eab-daaa-4696-961e-5296658a126c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:34:50.797156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f6eab-daaa-4696-961e-5296658a126c service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:34:50.807466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 737/1470] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:34:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:00.720303 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=17 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:00.802495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=17 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:24.078391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88c0f6-6c8c-4fc0-9cb1-9eb801710c31 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:24.079012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88c0f6-6c8c-4fc0-9cb1-9eb801710c31 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:24.079012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88c0f6-6c8c-4fc0-9cb1-9eb801710c31 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:24.080333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 734/1471] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:35:24.086353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf95a6-4e18-49cf-8932-71d74e1c4e38 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:24.086833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf95a6-4e18-49cf-8932-71d74e1c4e38 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:24.086930 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf95a6-4e18-49cf-8932-71d74e1c4e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:24.105101 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-36cf95a6-4e18-49cf-8932-71d74e1c4e38 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:24.106320 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-36cf95a6-4e18-49cf-8932-71d74e1c4e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:24.114337 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 738/1472] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:24.162465 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee048c7d-2ed9-4cbd-a08d-f91ea9a27a5d None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:24.162758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee048c7d-2ed9-4cbd-a08d-f91ea9a27a5d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:24.163037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee048c7d-2ed9-4cbd-a08d-f91ea9a27a5d None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:24.164427 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 735/1473] 10.222.0.64 () {54 vars in 962 bytes} [Mon May 23 15:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:35:24.169345 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc5517-f8e7-4981-a27b-427e962984ab None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:24.169646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc5517-f8e7-4981-a27b-427e962984ab None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:24.169934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc5517-f8e7-4981-a27b-427e962984ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:24.187818 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-77cc5517-f8e7-4981-a27b-427e962984ab None None] MFA Rules not processed for user `45ee7a6c342548f98e7dc73f4f71a66d`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:24.189007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-77cc5517-f8e7-4981-a27b-427e962984ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:24.196038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 739/1474] 10.222.0.64 () {58 vars in 1059 bytes} [Mon May 23 15:35:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:24.203685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee048c7d-2ed9-4cbd-a08d-f91ea9a27a5d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:24.214238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee048c7d-2ed9-4cbd-a08d-f91ea9a27a5d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:24.217715 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dda1bc-e4ef-49ad-8f2d-49a25d496a14 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee7a6c342548f98e7dc73f4f71a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:24.218348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dda1bc-e4ef-49ad-8f2d-49a25d496a14 service glance] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:24.218634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dda1bc-e4ef-49ad-8f2d-49a25d496a14 service glance] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:24.218914 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dda1bc-e4ef-49ad-8f2d-49a25d496a14 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:24.228323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dda1bc-e4ef-49ad-8f2d-49a25d496a14 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:24.229901 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dda1bc-e4ef-49ad-8f2d-49a25d496a14 service glance] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:24.243188 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 736/1475] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:26.580888 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2d93a-f2e7-4d49-8f5e-9a5ffee84891 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:26.581478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2d93a-f2e7-4d49-8f5e-9a5ffee84891 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:26.581478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2d93a-f2e7-4d49-8f5e-9a5ffee84891 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:26.583560 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 740/1476] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:35:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:35:26.589236 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae6822-b3da-4391-84d6-300dab0e11a2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:26.589688 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae6822-b3da-4391-84d6-300dab0e11a2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:26.589807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae6822-b3da-4391-84d6-300dab0e11a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:26.607293 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-54ae6822-b3da-4391-84d6-300dab0e11a2 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:26.608477 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-54ae6822-b3da-4391-84d6-300dab0e11a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:26.616170 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 737/1477] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:26.640025 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d2d93a-f2e7-4d49-8f5e-9a5ffee84891 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:26.652795 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d2d93a-f2e7-4d49-8f5e-9a5ffee84891 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:26.657166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e0605-df3c-4ebb-b15f-a9f85ade95eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:26.657811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e0605-df3c-4ebb-b15f-a9f85ade95eb admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:26.658095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e0605-df3c-4ebb-b15f-a9f85ade95eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:26.658372 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e0605-df3c-4ebb-b15f-a9f85ade95eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:26.659102 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e0605-df3c-4ebb-b15f-a9f85ade95eb admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:26.660893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: WARNING py.warnings [None req-a81e0605-df3c-4ebb-b15f-a9f85ade95eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:35:26.660893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: warnings.warn(msg) May 23 15:35:26.660893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: May 23 15:35:26.662173 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e0605-df3c-4ebb-b15f-a9f85ade95eb admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:26.681242 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 741/1478] 10.222.0.64 () {56 vars in 1108 bytes} [Mon May 23 15:35:26 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:35:28.936612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78a46e-a645-4c9e-a8a9-fd10b054faa9 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:28.937185 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78a46e-a645-4c9e-a8a9-fd10b054faa9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:28.937185 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78a46e-a645-4c9e-a8a9-fd10b054faa9 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:28.938559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 738/1479] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:35:28.944284 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721edd68-fcd6-4c88-b3a1-62bf8a2ffe17 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:28.944730 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721edd68-fcd6-4c88-b3a1-62bf8a2ffe17 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:28.944883 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721edd68-fcd6-4c88-b3a1-62bf8a2ffe17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:28.963651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-721edd68-fcd6-4c88-b3a1-62bf8a2ffe17 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:28.964959 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-721edd68-fcd6-4c88-b3a1-62bf8a2ffe17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:28.972677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 742/1480] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:29.016313 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78a46e-a645-4c9e-a8a9-fd10b054faa9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:29.030600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78a46e-a645-4c9e-a8a9-fd10b054faa9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:29.035980 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a450290-72fa-4a0b-becd-48fd695969dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:29.036735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a450290-72fa-4a0b-becd-48fd695969dc service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:29.037023 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a450290-72fa-4a0b-becd-48fd695969dc service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:29.037410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a450290-72fa-4a0b-becd-48fd695969dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:29.045393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a450290-72fa-4a0b-becd-48fd695969dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:29.046965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a450290-72fa-4a0b-becd-48fd695969dc service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:29.057795 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 739/1481] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:31.435307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb3675-9775-4c93-b56d-7ef3cbb9d388 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:31.435897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb3675-9775-4c93-b56d-7ef3cbb9d388 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:31.435897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb3675-9775-4c93-b56d-7ef3cbb9d388 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:31.437355 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 743/1482] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:35:31.443493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b55bbc-b7be-4518-ab7f-a7fa50cda3ba None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:31.443930 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b55bbc-b7be-4518-ab7f-a7fa50cda3ba None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:31.444043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b55bbc-b7be-4518-ab7f-a7fa50cda3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:31.462574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-25b55bbc-b7be-4518-ab7f-a7fa50cda3ba None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:31.463775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-25b55bbc-b7be-4518-ab7f-a7fa50cda3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:31.471686 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 740/1483] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:35:31.510833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb3675-9775-4c93-b56d-7ef3cbb9d388 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:31.522220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb3675-9775-4c93-b56d-7ef3cbb9d388 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:31.525752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59905442-5632-47e6-bf99-869ffc670abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:31.526391 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59905442-5632-47e6-bf99-869ffc670abf service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:31.526682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59905442-5632-47e6-bf99-869ffc670abf service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:31.526969 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59905442-5632-47e6-bf99-869ffc670abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:31.536244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59905442-5632-47e6-bf99-869ffc670abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:31.537744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59905442-5632-47e6-bf99-869ffc670abf service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:31.547214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 744/1484] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:35:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:33.821952 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b76032-07eb-435f-9604-60dd1c5f4682 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:33.822514 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b76032-07eb-435f-9604-60dd1c5f4682 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:33.822514 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b76032-07eb-435f-9604-60dd1c5f4682 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:33.823900 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 741/1485] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:35:33.829137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1832d7a-eee3-4dcf-92ac-034c856f295b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:33.829544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1832d7a-eee3-4dcf-92ac-034c856f295b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:33.829705 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1832d7a-eee3-4dcf-92ac-034c856f295b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:33.848761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c1832d7a-eee3-4dcf-92ac-034c856f295b None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:33.850020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c1832d7a-eee3-4dcf-92ac-034c856f295b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:33.857631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 745/1486] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:33.895134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b76032-07eb-435f-9604-60dd1c5f4682 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:33.906632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b76032-07eb-435f-9604-60dd1c5f4682 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:33.910160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8751d-d465-4973-8d1a-2120052ba9d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:33.911337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8751d-d465-4973-8d1a-2120052ba9d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:33.911624 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8751d-d465-4973-8d1a-2120052ba9d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:33.911912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8751d-d465-4973-8d1a-2120052ba9d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:33.920766 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8751d-d465-4973-8d1a-2120052ba9d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:33.922241 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8751d-d465-4973-8d1a-2120052ba9d9 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:33.932650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 742/1487] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:35:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:36.284730 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36941f8d-7cb7-4829-b939-0f29216257a7 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:36.285912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36941f8d-7cb7-4829-b939-0f29216257a7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:36.285912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36941f8d-7cb7-4829-b939-0f29216257a7 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:36.287207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 746/1488] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:35:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:35:36.292376 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23529fc-6274-49d7-80f8-8f9de1ccc128 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:36.292840 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23529fc-6274-49d7-80f8-8f9de1ccc128 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:36.292951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23529fc-6274-49d7-80f8-8f9de1ccc128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:36.311429 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b23529fc-6274-49d7-80f8-8f9de1ccc128 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:36.312638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b23529fc-6274-49d7-80f8-8f9de1ccc128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:36.320478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 743/1489] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:35:36.359772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36941f8d-7cb7-4829-b939-0f29216257a7 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:36.371027 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36941f8d-7cb7-4829-b939-0f29216257a7 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:36.374531 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6067517-f777-4dc6-842a-90553a87d133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:36.375173 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6067517-f777-4dc6-842a-90553a87d133 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:36.375461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6067517-f777-4dc6-842a-90553a87d133 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:36.375742 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6067517-f777-4dc6-842a-90553a87d133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:36.385494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6067517-f777-4dc6-842a-90553a87d133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:36.386935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6067517-f777-4dc6-842a-90553a87d133 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:36.397154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 747/1490] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:35:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:39.026120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05fc1d-cedc-4661-a197-674649edba79 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:39.026666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05fc1d-cedc-4661-a197-674649edba79 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:39.026666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05fc1d-cedc-4661-a197-674649edba79 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:39.027954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 744/1491] 10.222.0.64 () {54 vars in 926 bytes} [Mon May 23 15:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:35:39.033122 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134dae7-b7cd-40d1-9433-f72bfe6a080c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:39.033562 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134dae7-b7cd-40d1-9433-f72bfe6a080c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:39.033678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134dae7-b7cd-40d1-9433-f72bfe6a080c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:39.051777 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d134dae7-b7cd-40d1-9433-f72bfe6a080c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:39.052895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d134dae7-b7cd-40d1-9433-f72bfe6a080c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:39.060601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 748/1492] 10.222.0.64 () {58 vars in 1023 bytes} [Mon May 23 15:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:35:39.101899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05fc1d-cedc-4661-a197-674649edba79 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:39.109563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05fc1d-cedc-4661-a197-674649edba79 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:39.113855 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ba2d2-c9d2-4dab-9870-5a12401df11d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:39.114773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ba2d2-c9d2-4dab-9870-5a12401df11d service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:39.115056 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ba2d2-c9d2-4dab-9870-5a12401df11d service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:39.115335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ba2d2-c9d2-4dab-9870-5a12401df11d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:39.124122 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ba2d2-c9d2-4dab-9870-5a12401df11d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:39.125574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ba2d2-c9d2-4dab-9870-5a12401df11d service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:39.134978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 745/1493] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:35:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:35:49.045909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=33 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:35:49.130818 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=33 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:30.991927 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61e94f-4797-4004-853f-b12c4e3f41a6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.992427 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61e94f-4797-4004-853f-b12c4e3f41a6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.992427 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61e94f-4797-4004-853f-b12c4e3f41a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.010360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0c61e94f-4797-4004-853f-b12c4e3f41a6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:31.011575 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0c61e94f-4797-4004-853f-b12c4e3f41a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:31.019498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 749/1494] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:31.028495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b2147-dcbd-4a66-bdb6-74b152007e1c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.029511 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b2147-dcbd-4a66-bdb6-74b152007e1c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.029687 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b2147-dcbd-4a66-bdb6-74b152007e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.048007 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ec4b2147-dcbd-4a66-bdb6-74b152007e1c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:31.049643 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ec4b2147-dcbd-4a66-bdb6-74b152007e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:31.057392 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 746/1495] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:31.061881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61e94f-4797-4004-853f-b12c4e3f41a6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.071373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61e94f-4797-4004-853f-b12c4e3f41a6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.076040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ddbe7-bab9-4ff4-b49c-723c7bca3be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.076646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ddbe7-bab9-4ff4-b49c-723c7bca3be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.076935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ddbe7-bab9-4ff4-b49c-723c7bca3be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.077247 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ddbe7-bab9-4ff4-b49c-723c7bca3be5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.077979 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ddbe7-bab9-4ff4-b49c-723c7bca3be5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:31.078294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ddbe7-bab9-4ff4-b49c-723c7bca3be5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.079316 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: WARNING py.warnings [None req-163ddbe7-bab9-4ff4-b49c-723c7bca3be5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:37:31.079316 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: warnings.warn(msg) May 23 15:37:31.079316 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: May 23 15:37:31.080221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ddbe7-bab9-4ff4-b49c-723c7bca3be5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.092156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 750/1496] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:31.100147 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b2147-dcbd-4a66-bdb6-74b152007e1c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.110369 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b2147-dcbd-4a66-bdb6-74b152007e1c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.114947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3335fb-aee4-4af6-a604-3e053568b2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.115592 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3335fb-aee4-4af6-a604-3e053568b2a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.115904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3335fb-aee4-4af6-a604-3e053568b2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.116187 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3335fb-aee4-4af6-a604-3e053568b2a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.116938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3335fb-aee4-4af6-a604-3e053568b2a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.118699 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-dd3335fb-aee4-4af6-a604-3e053568b2a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:37:31.118699 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: warnings.warn(msg) May 23 15:37:31.118699 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: May 23 15:37:31.119936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3335fb-aee4-4af6-a604-3e053568b2a3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.246922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 747/1497] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:37:31 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:31.251284 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ddbe7-bab9-4ff4-b49c-723c7bca3be5 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.262267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ddbe7-bab9-4ff4-b49c-723c7bca3be5 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.319552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b16023-a9ef-4070-8b42-f2d468237dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.320584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b16023-a9ef-4070-8b42-f2d468237dcc admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.321038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b16023-a9ef-4070-8b42-f2d468237dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.321504 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b16023-a9ef-4070-8b42-f2d468237dcc admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.322813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b16023-a9ef-4070-8b42-f2d468237dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.324386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: WARNING py.warnings [None req-d8b16023-a9ef-4070-8b42-f2d468237dcc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:37:31.324386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: warnings.warn(msg) May 23 15:37:31.324386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: May 23 15:37:31.326405 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b16023-a9ef-4070-8b42-f2d468237dcc admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.470151 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 751/1498] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:37:31 2022] POST /identity/v3/users => generated 290 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:31.474327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3335fb-aee4-4af6-a604-3e053568b2a3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.484679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3335fb-aee4-4af6-a604-3e053568b2a3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.489340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424814d4-caa0-4ded-bc71-7def2267f128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.490010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424814d4-caa0-4ded-bc71-7def2267f128 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.490303 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424814d4-caa0-4ded-bc71-7def2267f128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.490585 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424814d4-caa0-4ded-bc71-7def2267f128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.491311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424814d4-caa0-4ded-bc71-7def2267f128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.492311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-424814d4-caa0-4ded-bc71-7def2267f128 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:37:31.492311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: warnings.warn(msg) May 23 15:37:31.492311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: May 23 15:37:31.493225 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424814d4-caa0-4ded-bc71-7def2267f128 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.506188 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 748/1499] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:31.510148 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b16023-a9ef-4070-8b42-f2d468237dcc admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.520721 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b16023-a9ef-4070-8b42-f2d468237dcc admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.525348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3517626-2e70-4b9b-8522-f9fd809e9b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.526009 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3517626-2e70-4b9b-8522-f9fd809e9b7b admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.526294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3517626-2e70-4b9b-8522-f9fd809e9b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.526573 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3517626-2e70-4b9b-8522-f9fd809e9b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.527298 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3517626-2e70-4b9b-8522-f9fd809e9b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.528367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: WARNING py.warnings [None req-b3517626-2e70-4b9b-8522-f9fd809e9b7b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:37:31.528367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: warnings.warn(msg) May 23 15:37:31.528367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: May 23 15:37:31.530032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3517626-2e70-4b9b-8522-f9fd809e9b7b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.542714 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 752/1500] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:37:31.546762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424814d4-caa0-4ded-bc71-7def2267f128 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.556843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424814d4-caa0-4ded-bc71-7def2267f128 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.561455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00736ba-5190-4f84-b3d1-fd36976f92dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.562338 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00736ba-5190-4f84-b3d1-fd36976f92dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.568329 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00736ba-5190-4f84-b3d1-fd36976f92dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.568329 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00736ba-5190-4f84-b3d1-fd36976f92dc admin admin] PATH_INFO: `/v3/projects/7fd2eab28ced44ca85b62103361d1ef4/users/ebd1b7d2f76640c383d3f33b983f350d/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.578378 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00736ba-5190-4f84-b3d1-fd36976f92dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fd2eab28ced44ca85b62103361d1ef4, user_id=ebd1b7d2f76640c383d3f33b983f350d, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.579409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-f00736ba-5190-4f84-b3d1-fd36976f92dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:37:31.579409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: warnings.warn(msg) May 23 15:37:31.579409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: May 23 15:37:31.581399 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00736ba-5190-4f84-b3d1-fd36976f92dc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.668862 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 749/1501] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:37:31 2022] PUT /identity/v3/projects/7fd2eab28ced44ca85b62103361d1ef4/users/ebd1b7d2f76640c383d3f33b983f350d/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:31.681751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91f713-a4fc-4caf-9719-58dacb22d21c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.682071 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91f713-a4fc-4caf-9719-58dacb22d21c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.682352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91f713-a4fc-4caf-9719-58dacb22d21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.838830 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ca91f713-a4fc-4caf-9719-58dacb22d21c None None] MFA Rules not processed for user `ebd1b7d2f76640c383d3f33b983f350d`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:31.840081 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ca91f713-a4fc-4caf-9719-58dacb22d21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:31.903066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 753/1502] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:37:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:31.916375 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178354a3-26a1-4d3d-9dd5-d91e17284b09 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.916673 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178354a3-26a1-4d3d-9dd5-d91e17284b09 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.916958 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178354a3-26a1-4d3d-9dd5-d91e17284b09 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.918382 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 750/1503] 10.222.0.64 () {54 vars in 963 bytes} [Mon May 23 15:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:31.923466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469e1c8-0928-4e5c-abf4-be51fcf257f4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.923760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469e1c8-0928-4e5c-abf4-be51fcf257f4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.924040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469e1c8-0928-4e5c-abf4-be51fcf257f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.941817 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a469e1c8-0928-4e5c-abf4-be51fcf257f4 None None] MFA Rules not processed for user `8750b0450bf343f590c52b7d5c68d5df`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:31.942996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a469e1c8-0928-4e5c-abf4-be51fcf257f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:31.985105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:31.992377 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 754/1504] 10.222.0.64 () {58 vars in 1060 bytes} [Mon May 23 15:37:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:32.000517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178354a3-26a1-4d3d-9dd5-d91e17284b09 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.010952 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178354a3-26a1-4d3d-9dd5-d91e17284b09 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.015019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc728390-3f29-4206-823a-f48f487a3c05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.015676 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc728390-3f29-4206-823a-f48f487a3c05 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.015964 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc728390-3f29-4206-823a-f48f487a3c05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.016243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc728390-3f29-4206-823a-f48f487a3c05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.024293 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc728390-3f29-4206-823a-f48f487a3c05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:32.040729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc728390-3f29-4206-823a-f48f487a3c05 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.040729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 751/1505] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:37:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:32.079010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed60cd-e987-42e3-ae60-6336000c055d None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.079298 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed60cd-e987-42e3-ae60-6336000c055d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.079574 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed60cd-e987-42e3-ae60-6336000c055d None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.080853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 755/1506] 10.222.0.64 () {52 vars in 829 bytes} [Mon May 23 15:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:32.084590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc728390-3f29-4206-823a-f48f487a3c05 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.147259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc728390-3f29-4206-823a-f48f487a3c05 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.151688 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797cae77-77b3-43a6-9aeb-f3d5e8e0728a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.152558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cae77-77b3-43a6-9aeb-f3d5e8e0728a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.152849 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cae77-77b3-43a6-9aeb-f3d5e8e0728a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.153154 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cae77-77b3-43a6-9aeb-f3d5e8e0728a admin admin] PATH_INFO: `/v3/users/ebd1b7d2f76640c383d3f33b983f350d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.154380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797cae77-77b3-43a6-9aeb-f3d5e8e0728a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebd1b7d2f76640c383d3f33b983f350d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:32.155349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-797cae77-77b3-43a6-9aeb-f3d5e8e0728a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:37:32.155349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: warnings.warn(msg) May 23 15:37:32.155349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: May 23 15:37:32.156558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797cae77-77b3-43a6-9aeb-f3d5e8e0728a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.409968 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-797cae77-77b3-43a6-9aeb-f3d5e8e0728a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebd1b7d2f76640c383d3f33b983f350d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:32.430368 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-797cae77-77b3-43a6-9aeb-f3d5e8e0728a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebd1b7d2f76640c383d3f33b983f350d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:32.542422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-797cae77-77b3-43a6-9aeb-f3d5e8e0728a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebd1b7d2f76640c383d3f33b983f350d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:32.543054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-797cae77-77b3-43a6-9aeb-f3d5e8e0728a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebd1b7d2f76640c383d3f33b983f350d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:32.553232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:32.557636 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-797cae77-77b3-43a6-9aeb-f3d5e8e0728a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebd1b7d2f76640c383d3f33b983f350d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:32.564064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 752/1507] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:37:32 2022] DELETE /identity/v3/users/ebd1b7d2f76640c383d3f33b983f350d => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:32.568460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e5dcc-95c7-4e0e-b329-823d153c9b2a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.568757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e5dcc-95c7-4e0e-b329-823d153c9b2a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.569040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e5dcc-95c7-4e0e-b329-823d153c9b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.587004 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-4d0e5dcc-95c7-4e0e-b329-823d153c9b2a None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:32.588206 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4d0e5dcc-95c7-4e0e-b329-823d153c9b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:32.645809 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 756/1508] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:37:32.657190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797cae77-77b3-43a6-9aeb-f3d5e8e0728a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.658587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-797cae77-77b3-43a6-9aeb-f3d5e8e0728a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:32.715526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797cae77-77b3-43a6-9aeb-f3d5e8e0728a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.719779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a01a9-5a46-4b95-81ac-feddb4b2deb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.720433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a01a9-5a46-4b95-81ac-feddb4b2deb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.720716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a01a9-5a46-4b95-81ac-feddb4b2deb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.720993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a01a9-5a46-4b95-81ac-feddb4b2deb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.729740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a01a9-5a46-4b95-81ac-feddb4b2deb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:32.731161 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a01a9-5a46-4b95-81ac-feddb4b2deb9 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.741054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 753/1509] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:37:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:37:33.213640 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e5dcc-95c7-4e0e-b329-823d153c9b2a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:33.215156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4d0e5dcc-95c7-4e0e-b329-823d153c9b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:33.227924 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e5dcc-95c7-4e0e-b329-823d153c9b2a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:33.232663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ed029-a1d9-48c3-b94f-5292627bb553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:33.233580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ed029-a1d9-48c3-b94f-5292627bb553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.233863 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ed029-a1d9-48c3-b94f-5292627bb553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.234148 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ed029-a1d9-48c3-b94f-5292627bb553 admin admin] PATH_INFO: `/v3/projects/7fd2eab28ced44ca85b62103361d1ef4` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.235342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ed029-a1d9-48c3-b94f-5292627bb553 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fd2eab28ced44ca85b62103361d1ef4)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:33.236371 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: WARNING py.warnings [None req-8c9ed029-a1d9-48c3-b94f-5292627bb553 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:37:33.236371 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: warnings.warn(msg) May 23 15:37:33.236371 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: May 23 15:37:33.237604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ed029-a1d9-48c3-b94f-5292627bb553 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:33.369522 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8c9ed029-a1d9-48c3-b94f-5292627bb553 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fd2eab28ced44ca85b62103361d1ef4', 'request_id': 'req-8c9ed029-a1d9-48c3-b94f-5292627bb553'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:33.376623 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8c9ed029-a1d9-48c3-b94f-5292627bb553 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fd2eab28ced44ca85b62103361d1ef4', 'request_id': 'req-8c9ed029-a1d9-48c3-b94f-5292627bb553'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:33.391342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8c9ed029-a1d9-48c3-b94f-5292627bb553 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fd2eab28ced44ca85b62103361d1ef4', 'request_id': 'req-8c9ed029-a1d9-48c3-b94f-5292627bb553'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:33.622276 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8c9ed029-a1d9-48c3-b94f-5292627bb553 admin admin] The token cache is being invalidate because project 7fd2eab28ced44ca85b62103361d1ef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:37:33.622608 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8c9ed029-a1d9-48c3-b94f-5292627bb553 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:33.624356 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8c9ed029-a1d9-48c3-b94f-5292627bb553 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:37:33.626074 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 757/1510] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:37:33 2022] DELETE /identity/v3/projects/7fd2eab28ced44ca85b62103361d1ef4 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:42.737006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=29 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:43.619045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:19.200477 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447986c-8ed3-44ef-bff7-ed6519181f8d None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:19.202632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447986c-8ed3-44ef-bff7-ed6519181f8d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:19.202632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447986c-8ed3-44ef-bff7-ed6519181f8d None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:19.202632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:39:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:39:19.207425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce728d2-8bcb-4849-909b-980c120a12a4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:19.207862 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce728d2-8bcb-4849-909b-980c120a12a4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:19.207990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce728d2-8bcb-4849-909b-980c120a12a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:19.227438 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-dce728d2-8bcb-4849-909b-980c120a12a4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:19.228674 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-dce728d2-8bcb-4849-909b-980c120a12a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:19.289030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:39:19.462414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447986c-8ed3-44ef-bff7-ed6519181f8d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:19.464229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8447986c-8ed3-44ef-bff7-ed6519181f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:19.540190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447986c-8ed3-44ef-bff7-ed6519181f8d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:19.546693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950900cf-d82e-4521-add6-b71389a699d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:19.547347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950900cf-d82e-4521-add6-b71389a699d9 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:19.547638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950900cf-d82e-4521-add6-b71389a699d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:19.547956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950900cf-d82e-4521-add6-b71389a699d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:19.556059 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950900cf-d82e-4521-add6-b71389a699d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:19.557708 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950900cf-d82e-4521-add6-b71389a699d9 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:19.567628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 755/1513] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:39:20.436496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410860aa-1624-4a80-9ac5-a56af78c7253 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:20.437121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410860aa-1624-4a80-9ac5-a56af78c7253 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:20.437121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410860aa-1624-4a80-9ac5-a56af78c7253 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:20.438469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:39:20.443361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac15434-95fe-429f-ba3c-15801de9410d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:20.443814 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac15434-95fe-429f-ba3c-15801de9410d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:20.444116 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac15434-95fe-429f-ba3c-15801de9410d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:20.462542 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fac15434-95fe-429f-ba3c-15801de9410d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:20.472355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fac15434-95fe-429f-ba3c-15801de9410d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:20.472355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:39:20.621286 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410860aa-1624-4a80-9ac5-a56af78c7253 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:20.635459 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410860aa-1624-4a80-9ac5-a56af78c7253 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:20.641495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098306ad-c07c-4108-bd70-519393d878b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:20.642133 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098306ad-c07c-4108-bd70-519393d878b5 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:20.642423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098306ad-c07c-4108-bd70-519393d878b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:20.642706 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098306ad-c07c-4108-bd70-519393d878b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:20.650863 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098306ad-c07c-4108-bd70-519393d878b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:20.652478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098306ad-c07c-4108-bd70-519393d878b5 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:20.661788 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 760/1516] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:39:22.946115 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e07389-0f4d-4b14-b3fa-8553e2832cc9 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:22.946683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e07389-0f4d-4b14-b3fa-8553e2832cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:22.946683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e07389-0f4d-4b14-b3fa-8553e2832cc9 None None] PATH_INFO: `/v3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:22.948099 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 757/1517] 10.222.0.69 () {54 vars in 925 bytes} [Mon May 23 15:39:22 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:39:22.955826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b4daf-0c62-47df-80c1-7e48ac912eec None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:22.956249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b4daf-0c62-47df-80c1-7e48ac912eec None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:22.957932 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b4daf-0c62-47df-80c1-7e48ac912eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:22.974043 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ac8b4daf-0c62-47df-80c1-7e48ac912eec None None] MFA Rules not processed for user `04fbaeb7583a456d81cf79f6b101fb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:22.975288 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ac8b4daf-0c62-47df-80c1-7e48ac912eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:23.037007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 761/1518] 10.222.0.69 () {58 vars in 1015 bytes} [Mon May 23 15:39:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:39:30.457325 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=37 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:31.948146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d15cb4-45a2-4731-af0f-c54a25627bda None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:31.948907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d15cb4-45a2-4731-af0f-c54a25627bda None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:31.948907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d15cb4-45a2-4731-af0f-c54a25627bda None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:31.950727 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 758/1519] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:39:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:39:31.955593 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8a3b2-c9e0-413a-b82c-da16a2ae2e7f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:31.956010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8a3b2-c9e0-413a-b82c-da16a2ae2e7f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:31.956160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8a3b2-c9e0-413a-b82c-da16a2ae2e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:31.974393 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e4f8a3b2-c9e0-413a-b82c-da16a2ae2e7f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:31.975600 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e4f8a3b2-c9e0-413a-b82c-da16a2ae2e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:31.983948 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 762/1520] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:39:32.142159 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d15cb4-45a2-4731-af0f-c54a25627bda None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:32.157608 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d15cb4-45a2-4731-af0f-c54a25627bda None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:32.163115 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f5be9-370f-436c-b92d-50b2f3c9387b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:32.163768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f5be9-370f-436c-b92d-50b2f3c9387b service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:32.164066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f5be9-370f-436c-b92d-50b2f3c9387b service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:32.164347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f5be9-370f-436c-b92d-50b2f3c9387b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:32.173948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f5be9-370f-436c-b92d-50b2f3c9387b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:32.175595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f5be9-370f-436c-b92d-50b2f3c9387b service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:32.185832 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 759/1521] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:39:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:39:33.085804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bc452-9657-4143-9e2b-47c0fdeec76b None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:33.086362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bc452-9657-4143-9e2b-47c0fdeec76b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:33.086362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bc452-9657-4143-9e2b-47c0fdeec76b None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:33.087758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:39:33.093222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa436b-3e8f-4be8-bb93-ea0eb875f55b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:33.093666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa436b-3e8f-4be8-bb93-ea0eb875f55b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:33.093823 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa436b-3e8f-4be8-bb93-ea0eb875f55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:33.112168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-81fa436b-3e8f-4be8-bb93-ea0eb875f55b None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:33.113410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-81fa436b-3e8f-4be8-bb93-ea0eb875f55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:33.121259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:39:33.273431 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bc452-9657-4143-9e2b-47c0fdeec76b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:33.287866 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bc452-9657-4143-9e2b-47c0fdeec76b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:33.293858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16d504-6bdb-4dbb-a98f-a2bc8f62553f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:33.294486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16d504-6bdb-4dbb-a98f-a2bc8f62553f service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:33.294765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16d504-6bdb-4dbb-a98f-a2bc8f62553f service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:33.295288 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16d504-6bdb-4dbb-a98f-a2bc8f62553f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:33.303199 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16d504-6bdb-4dbb-a98f-a2bc8f62553f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:33.304773 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16d504-6bdb-4dbb-a98f-a2bc8f62553f service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:33.313898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 764/1524] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:39:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:39:34.426319 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34fd28-d66f-4637-b3ad-1d58efb87c7e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:34.426867 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34fd28-d66f-4637-b3ad-1d58efb87c7e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:34.426867 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34fd28-d66f-4637-b3ad-1d58efb87c7e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:34.428178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 761/1525] 10.222.0.64 () {54 vars in 960 bytes} [Mon May 23 15:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:39:34.433913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc010e1-0dfb-4420-9987-6334c40a287f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:34.434342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc010e1-0dfb-4420-9987-6334c40a287f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:34.434447 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc010e1-0dfb-4420-9987-6334c40a287f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:34.452704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-1fc010e1-0dfb-4420-9987-6334c40a287f None None] MFA Rules not processed for user `04fbaeb7583a456d81cf79f6b101fb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:34.453951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1fc010e1-0dfb-4420-9987-6334c40a287f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:34.462260 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 765/1526] 10.222.0.64 () {58 vars in 1057 bytes} [Mon May 23 15:39:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:39:34.471821 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34fd28-d66f-4637-b3ad-1d58efb87c7e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:34.486607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34fd28-d66f-4637-b3ad-1d58efb87c7e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:34.491526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b7093-4eb2-43a1-b2c9-6350d9e1b0b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fbaeb7583a456d81cf79f6b101fb1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:34.492165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b7093-4eb2-43a1-b2c9-6350d9e1b0b4 service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:34.492451 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b7093-4eb2-43a1-b2c9-6350d9e1b0b4 service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:34.492729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b7093-4eb2-43a1-b2c9-6350d9e1b0b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:34.501304 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b7093-4eb2-43a1-b2c9-6350d9e1b0b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:34.502908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b7093-4eb2-43a1-b2c9-6350d9e1b0b4 service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:34.513754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 762/1527] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:39:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:39:44.447504 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=74 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:44.508770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=15 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:44.610851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f237486-c1ec-4cbc-a0a8-92dea13467db None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:44.611143 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f237486-c1ec-4cbc-a0a8-92dea13467db None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:44.611430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f237486-c1ec-4cbc-a0a8-92dea13467db None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:44.612848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:39:44.617672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a200e3-f4dd-44a2-82df-05c8d9c89c39 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:44.617993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a200e3-f4dd-44a2-82df-05c8d9c89c39 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:44.618305 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a200e3-f4dd-44a2-82df-05c8d9c89c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:44.635992 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-68a200e3-f4dd-44a2-82df-05c8d9c89c39 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:44.637219 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-68a200e3-f4dd-44a2-82df-05c8d9c89c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:44.645106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:39:44.804345 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f237486-c1ec-4cbc-a0a8-92dea13467db None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:44.818546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f237486-c1ec-4cbc-a0a8-92dea13467db None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:44.823843 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a6113-13c6-41bb-b44e-70e7bdae0e88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:44.824485 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a6113-13c6-41bb-b44e-70e7bdae0e88 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:44.824777 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a6113-13c6-41bb-b44e-70e7bdae0e88 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:44.825074 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a6113-13c6-41bb-b44e-70e7bdae0e88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:44.833911 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a6113-13c6-41bb-b44e-70e7bdae0e88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:44.835510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a6113-13c6-41bb-b44e-70e7bdae0e88 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:44.845864 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 767/1530] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:39:45.731003 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4906f-b7eb-4c06-8944-363c5dc7a833 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:45.731520 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4906f-b7eb-4c06-8944-363c5dc7a833 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:45.731625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4906f-b7eb-4c06-8944-363c5dc7a833 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:45.733480 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:39:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:39:45.738273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b59b9b-7d9c-45e3-95ef-e429e7d6f0e7 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:45.738732 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b59b9b-7d9c-45e3-95ef-e429e7d6f0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:45.738862 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b59b9b-7d9c-45e3-95ef-e429e7d6f0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:45.758108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a6b59b9b-7d9c-45e3-95ef-e429e7d6f0e7 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:45.759357 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a6b59b9b-7d9c-45e3-95ef-e429e7d6f0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:45.767588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:39:45.924532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4906f-b7eb-4c06-8944-363c5dc7a833 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:45.939035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4906f-b7eb-4c06-8944-363c5dc7a833 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:45.944389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c1a71-8d55-4eb4-9691-8bab6952e821 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:45.945027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c1a71-8d55-4eb4-9691-8bab6952e821 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:45.945351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c1a71-8d55-4eb4-9691-8bab6952e821 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:45.945630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c1a71-8d55-4eb4-9691-8bab6952e821 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:45.954133 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c1a71-8d55-4eb4-9691-8bab6952e821 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:45.955664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c1a71-8d55-4eb4-9691-8bab6952e821 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:45.964846 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 765/1533] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:39:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:39:50.133806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353b5dd-dfe5-4255-952c-5b23c57f6bef None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:50.134440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353b5dd-dfe5-4255-952c-5b23c57f6bef None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:50.134440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353b5dd-dfe5-4255-952c-5b23c57f6bef None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:50.135767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:39:50.141052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4b424-9738-412a-a498-74e833d98e18 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:50.141512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4b424-9738-412a-a498-74e833d98e18 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:50.141646 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4b424-9738-412a-a498-74e833d98e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:50.159514 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-bde4b424-9738-412a-a498-74e833d98e18 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:50.160739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bde4b424-9738-412a-a498-74e833d98e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:50.168647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:39:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:39:50.207013 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353b5dd-dfe5-4255-952c-5b23c57f6bef None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:50.208741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c353b5dd-dfe5-4255-952c-5b23c57f6bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:50.267474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353b5dd-dfe5-4255-952c-5b23c57f6bef None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:50.271245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ce99a-ef73-40b7-b59a-bb409df39a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:50.271896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ce99a-ef73-40b7-b59a-bb409df39a49 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:50.272187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ce99a-ef73-40b7-b59a-bb409df39a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:50.272468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ce99a-ef73-40b7-b59a-bb409df39a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:50.281549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ce99a-ef73-40b7-b59a-bb409df39a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:50.283010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ce99a-ef73-40b7-b59a-bb409df39a49 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:50.292199 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 770/1536] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:39:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:39:52.524524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d43c46-181a-4394-a194-7e80ad78f209 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:52.525092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d43c46-181a-4394-a194-7e80ad78f209 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:52.525222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d43c46-181a-4394-a194-7e80ad78f209 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:52.526531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:39:52.532571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7b72d-513d-45dd-8333-ca8bbd00ca74 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:52.533017 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7b72d-513d-45dd-8333-ca8bbd00ca74 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:52.533212 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7b72d-513d-45dd-8333-ca8bbd00ca74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:52.551941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3da7b72d-513d-45dd-8333-ca8bbd00ca74 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:52.553275 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3da7b72d-513d-45dd-8333-ca8bbd00ca74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:52.561028 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:39:52.596785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d43c46-181a-4394-a194-7e80ad78f209 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:52.607836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d43c46-181a-4394-a194-7e80ad78f209 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:52.611454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08a772-3523-4a7c-8717-fd971cbdafdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:52.612202 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08a772-3523-4a7c-8717-fd971cbdafdb service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:52.612488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08a772-3523-4a7c-8717-fd971cbdafdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:52.612765 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08a772-3523-4a7c-8717-fd971cbdafdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:52.623113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08a772-3523-4a7c-8717-fd971cbdafdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:52.624580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08a772-3523-4a7c-8717-fd971cbdafdb service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:52.634258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 768/1539] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:39:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:39:56.571674 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95d3db-da45-4966-8f01-41b8c213d2b8 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:56.572195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95d3db-da45-4966-8f01-41b8c213d2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:56.572301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95d3db-da45-4966-8f01-41b8c213d2b8 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:56.573633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 772/1540] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:39:56.579097 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64d4ff-7804-4058-b05c-944b747acfa7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:56.579543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64d4ff-7804-4058-b05c-944b747acfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:56.579675 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64d4ff-7804-4058-b05c-944b747acfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:56.598040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3e64d4ff-7804-4058-b05c-944b747acfa7 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:56.599970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3e64d4ff-7804-4058-b05c-944b747acfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:56.607741 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 769/1541] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:39:56.883105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95d3db-da45-4966-8f01-41b8c213d2b8 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:56.884814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8d95d3db-da45-4966-8f01-41b8c213d2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:56.941976 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95d3db-da45-4966-8f01-41b8c213d2b8 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:56.945516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6038b-7025-4c51-9d6c-e799d11f4aa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:56.946163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6038b-7025-4c51-9d6c-e799d11f4aa3 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:56.946450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6038b-7025-4c51-9d6c-e799d11f4aa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:56.946732 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6038b-7025-4c51-9d6c-e799d11f4aa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:56.955812 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6038b-7025-4c51-9d6c-e799d11f4aa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:56.957226 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6038b-7025-4c51-9d6c-e799d11f4aa3 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:56.967079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 773/1542] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:39:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:39:59.219840 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429a2ed-18fe-4034-ad1f-c17ed67b44d7 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:59.220414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429a2ed-18fe-4034-ad1f-c17ed67b44d7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:59.220414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429a2ed-18fe-4034-ad1f-c17ed67b44d7 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:59.221801 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 770/1543] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:39:59.227128 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c14c1f-d654-4c42-90fd-2e979ee6d177 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:59.227555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c14c1f-d654-4c42-90fd-2e979ee6d177 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:59.227718 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c14c1f-d654-4c42-90fd-2e979ee6d177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:59.245478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-48c14c1f-d654-4c42-90fd-2e979ee6d177 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:59.247274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-48c14c1f-d654-4c42-90fd-2e979ee6d177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:59.255609 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 774/1544] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:39:59.523524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429a2ed-18fe-4034-ad1f-c17ed67b44d7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:59.525225 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7429a2ed-18fe-4034-ad1f-c17ed67b44d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:59.538618 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429a2ed-18fe-4034-ad1f-c17ed67b44d7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:59.542054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1113c2c-2330-42ff-a8c9-bf23a52b3008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:59.542686 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1113c2c-2330-42ff-a8c9-bf23a52b3008 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:59.542971 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1113c2c-2330-42ff-a8c9-bf23a52b3008 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:59.543250 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1113c2c-2330-42ff-a8c9-bf23a52b3008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:59.552271 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1113c2c-2330-42ff-a8c9-bf23a52b3008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:59.553709 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1113c2c-2330-42ff-a8c9-bf23a52b3008 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:59.563259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 771/1545] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:39:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:40:09.240262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=67 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:09.558718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=27 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:29.807162 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449d8fb-ceca-4127-bc38-41f676bbdfcc None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:29.807829 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449d8fb-ceca-4127-bc38-41f676bbdfcc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:29.807829 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449d8fb-ceca-4127-bc38-41f676bbdfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:29.815333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd12ab-6dbb-4006-8f76-c5021feb63d7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:29.825745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd12ab-6dbb-4006-8f76-c5021feb63d7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:29.825745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd12ab-6dbb-4006-8f76-c5021feb63d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:29.858051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e5cd12ab-6dbb-4006-8f76-c5021feb63d7 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:29.869840 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a449d8fb-ceca-4127-bc38-41f676bbdfcc None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:29.871084 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a449d8fb-ceca-4127-bc38-41f676bbdfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:29.872995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e5cd12ab-6dbb-4006-8f76-c5021feb63d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:29.892144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 772/1546] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:29.893525 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261f37a-c515-4436-bcf8-c0c851fdf8b6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:29.893759 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 775/1547] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:29.893880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261f37a-c515-4436-bcf8-c0c851fdf8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:29.894093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261f37a-c515-4436-bcf8-c0c851fdf8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:29.895977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc297b-66b3-4047-8e1d-274cebb79a5d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:29.896275 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc297b-66b3-4047-8e1d-274cebb79a5d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:29.896558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc297b-66b3-4047-8e1d-274cebb79a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:29.920956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a261f37a-c515-4436-bcf8-c0c851fdf8b6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:29.922278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a261f37a-c515-4436-bcf8-c0c851fdf8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:29.935185 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 773/1548] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:29.937125 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc851e-a158-4d30-99d7-bd866f0f756f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:29.937599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc851e-a158-4d30-99d7-bd866f0f756f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:29.937931 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc851e-a158-4d30-99d7-bd866f0f756f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:29.939055 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-85fc297b-66b3-4047-8e1d-274cebb79a5d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:29.940310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-85fc297b-66b3-4047-8e1d-274cebb79a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:29.960039 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fadc851e-a158-4d30-99d7-bd866f0f756f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:29.961439 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fadc851e-a158-4d30-99d7-bd866f0f756f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:29.962780 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 776/1549] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:29.969085 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed9964-7395-496b-aa27-e6647038a9a6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:29.969085 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed9964-7395-496b-aa27-e6647038a9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:29.969085 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed9964-7395-496b-aa27-e6647038a9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:29.981222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 774/1550] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:29.989126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee11bfc-ca1b-4c4d-abf7-b59e9c32e765 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:29.989126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee11bfc-ca1b-4c4d-abf7-b59e9c32e765 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:29.989126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee11bfc-ca1b-4c4d-abf7-b59e9c32e765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:29.990553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-5fed9964-7395-496b-aa27-e6647038a9a6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:29.991889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5fed9964-7395-496b-aa27-e6647038a9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:30.005779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 777/1551] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:30.010480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bd262-0de0-43a0-adb9-11949966ee40 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.010776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bd262-0de0-43a0-adb9-11949966ee40 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.011057 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bd262-0de0-43a0-adb9-11949966ee40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.037616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4ee11bfc-ca1b-4c4d-abf7-b59e9c32e765 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:30.038866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4ee11bfc-ca1b-4c4d-abf7-b59e9c32e765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:30.046249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-857bd262-0de0-43a0-adb9-11949966ee40 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:30.047448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-857bd262-0de0-43a0-adb9-11949966ee40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:30.050127 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 775/1552] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:30.051487 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1396e-4938-4cd7-b034-0d5d13af99ed None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.051792 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1396e-4938-4cd7-b034-0d5d13af99ed None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.052074 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1396e-4938-4cd7-b034-0d5d13af99ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.056236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 778/1553] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:30.056536 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bd262-0de0-43a0-adb9-11949966ee40 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.074361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6eb1396e-4938-4cd7-b034-0d5d13af99ed None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:30.074588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bd262-0de0-43a0-adb9-11949966ee40 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.075720 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6eb1396e-4938-4cd7-b034-0d5d13af99ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:30.079583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f8555-f9c5-4b65-8c5a-bee868e92c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.080450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f8555-f9c5-4b65-8c5a-bee868e92c2e admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.080740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f8555-f9c5-4b65-8c5a-bee868e92c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.081020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f8555-f9c5-4b65-8c5a-bee868e92c2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.081832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f8555-f9c5-4b65-8c5a-bee868e92c2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:41:30.082147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f8555-f9c5-4b65-8c5a-bee868e92c2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.083780 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f8555-f9c5-4b65-8c5a-bee868e92c2e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:30.086285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 776/1554] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:30.087644 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1396e-4938-4cd7-b034-0d5d13af99ed None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.097455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 779/1555] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:41:30.098763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd7d8a-aeff-45a9-a9b9-fc7ca9d659a4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.099061 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd7d8a-aeff-45a9-a9b9-fc7ca9d659a4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.099343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd7d8a-aeff-45a9-a9b9-fc7ca9d659a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.106539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1396e-4938-4cd7-b034-0d5d13af99ed None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.111406 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5af01d-76d5-401c-958f-90982f474b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.112114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5af01d-76d5-401c-958f-90982f474b07 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.112401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5af01d-76d5-401c-958f-90982f474b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.112682 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5af01d-76d5-401c-958f-90982f474b07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.113435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5af01d-76d5-401c-958f-90982f474b07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:41:30.117677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5af01d-76d5-401c-958f-90982f474b07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.117677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-ba5af01d-76d5-401c-958f-90982f474b07 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:41:30.117677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: warnings.warn(msg) May 23 15:41:30.117677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: May 23 15:41:30.117677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5af01d-76d5-401c-958f-90982f474b07 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:30.118515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c8fd7d8a-aeff-45a9-a9b9-fc7ca9d659a4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:30.119749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c8fd7d8a-aeff-45a9-a9b9-fc7ca9d659a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:30.128192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 777/1556] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:41:30.128995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 780/1557] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:30.129476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753025df-de3c-4fac-903d-38dc53aa960e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.129626 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd7d8a-aeff-45a9-a9b9-fc7ca9d659a4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.129762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753025df-de3c-4fac-903d-38dc53aa960e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.130039 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753025df-de3c-4fac-903d-38dc53aa960e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.147351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd7d8a-aeff-45a9-a9b9-fc7ca9d659a4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.150676 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-753025df-de3c-4fac-903d-38dc53aa960e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:30.151865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3296a71-c3db-48f7-8927-67c2dafa53dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.152055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-753025df-de3c-4fac-903d-38dc53aa960e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:30.152515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3296a71-c3db-48f7-8927-67c2dafa53dc admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.152798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3296a71-c3db-48f7-8927-67c2dafa53dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.153148 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3296a71-c3db-48f7-8927-67c2dafa53dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.153823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3296a71-c3db-48f7-8927-67c2dafa53dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:41:30.154126 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3296a71-c3db-48f7-8927-67c2dafa53dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.156096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3296a71-c3db-48f7-8927-67c2dafa53dc admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:30.168681 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 778/1558] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:30.168681 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9970dd-3371-4985-842a-6590d62d2f7a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.168681 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9970dd-3371-4985-842a-6590d62d2f7a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.168681 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9970dd-3371-4985-842a-6590d62d2f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.175737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 781/1559] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:41:30.175737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3296a71-c3db-48f7-8927-67c2dafa53dc admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.181144 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3296a71-c3db-48f7-8927-67c2dafa53dc admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.185715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90254c3b-24b5-49d4-b23f-d1b8063939d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.186365 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90254c3b-24b5-49d4-b23f-d1b8063939d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.187821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90254c3b-24b5-49d4-b23f-d1b8063939d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.187821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90254c3b-24b5-49d4-b23f-d1b8063939d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.188444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90254c3b-24b5-49d4-b23f-d1b8063939d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:41:30.188751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90254c3b-24b5-49d4-b23f-d1b8063939d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.189831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8e9970dd-3371-4985-842a-6590d62d2f7a None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:30.190418 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90254c3b-24b5-49d4-b23f-d1b8063939d9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:30.191070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8e9970dd-3371-4985-842a-6590d62d2f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:30.199071 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 779/1560] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:30.200365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41cea0-2418-4975-a82e-88265a4f0c2e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.200655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41cea0-2418-4975-a82e-88265a4f0c2e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.200940 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41cea0-2418-4975-a82e-88265a4f0c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.210940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 782/1561] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:41:30.210940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90254c3b-24b5-49d4-b23f-d1b8063939d9 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.216309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90254c3b-24b5-49d4-b23f-d1b8063939d9 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.219763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6e41cea0-2418-4975-a82e-88265a4f0c2e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:30.220970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6e41cea0-2418-4975-a82e-88265a4f0c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:30.221205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7a286-6495-42eb-bfa3-bb5d86e54473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.221866 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7a286-6495-42eb-bfa3-bb5d86e54473 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.222157 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7a286-6495-42eb-bfa3-bb5d86e54473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.222521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7a286-6495-42eb-bfa3-bb5d86e54473 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.223217 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7a286-6495-42eb-bfa3-bb5d86e54473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.224222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: WARNING py.warnings [None req-bec7a286-6495-42eb-bfa3-bb5d86e54473 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:41:30.224222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: warnings.warn(msg) May 23 15:41:30.224222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: May 23 15:41:30.225427 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7a286-6495-42eb-bfa3-bb5d86e54473 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:30.229631 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 780/1562] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:30.230435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41cea0-2418-4975-a82e-88265a4f0c2e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.240527 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41cea0-2418-4975-a82e-88265a4f0c2e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.245048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b41c43-3a16-438c-a15c-a5e9fbe8194b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.245730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b41c43-3a16-438c-a15c-a5e9fbe8194b admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.246018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b41c43-3a16-438c-a15c-a5e9fbe8194b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.246300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b41c43-3a16-438c-a15c-a5e9fbe8194b admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.246993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b41c43-3a16-438c-a15c-a5e9fbe8194b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.248887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b41c43-3a16-438c-a15c-a5e9fbe8194b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:30.318003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 783/1563] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:30.319411 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728fad2-aecf-4e3f-8b2e-9342557a1071 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.319693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728fad2-aecf-4e3f-8b2e-9342557a1071 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.319968 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728fad2-aecf-4e3f-8b2e-9342557a1071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.336923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e728fad2-aecf-4e3f-8b2e-9342557a1071 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:30.338173 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e728fad2-aecf-4e3f-8b2e-9342557a1071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:30.347060 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 781/1564] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:30.348651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4b0ab-7347-4df3-a168-f7cff113e9c7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.348984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4b0ab-7347-4df3-a168-f7cff113e9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.349324 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4b0ab-7347-4df3-a168-f7cff113e9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.367576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c9f4b0ab-7347-4df3-a168-f7cff113e9c7 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:30.368796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c9f4b0ab-7347-4df3-a168-f7cff113e9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:30.406104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 784/1565] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:30.406104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728fad2-aecf-4e3f-8b2e-9342557a1071 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.416402 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728fad2-aecf-4e3f-8b2e-9342557a1071 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.430707 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 782/1566] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:30.431525 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4b0ab-7347-4df3-a168-f7cff113e9c7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.441209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4b0ab-7347-4df3-a168-f7cff113e9c7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.445686 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7107143-b421-43b6-a52f-513c406c3cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.446330 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7107143-b421-43b6-a52f-513c406c3cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.446615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7107143-b421-43b6-a52f-513c406c3cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.446894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7107143-b421-43b6-a52f-513c406c3cf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.447571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7107143-b421-43b6-a52f-513c406c3cf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.449526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7107143-b421-43b6-a52f-513c406c3cf8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:30.477916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7f816-ffdb-4c4d-b362-e43cba77277e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.478819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7f816-ffdb-4c4d-b362-e43cba77277e admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.479244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7f816-ffdb-4c4d-b362-e43cba77277e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.479659 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7f816-ffdb-4c4d-b362-e43cba77277e admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.480763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7f816-ffdb-4c4d-b362-e43cba77277e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.484556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7f816-ffdb-4c4d-b362-e43cba77277e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:30.495824 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:30.529280 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 783/1567] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:30.530184 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7107143-b421-43b6-a52f-513c406c3cf8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.582250 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:30.593203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7107143-b421-43b6-a52f-513c406c3cf8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.597737 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000f31c-3c36-4199-b3b7-6fce547a63fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.598393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000f31c-3c36-4199-b3b7-6fce547a63fc admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.598680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000f31c-3c36-4199-b3b7-6fce547a63fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.598954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000f31c-3c36-4199-b3b7-6fce547a63fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.599668 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000f31c-3c36-4199-b3b7-6fce547a63fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:41:30.599973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000f31c-3c36-4199-b3b7-6fce547a63fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.601645 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000f31c-3c36-4199-b3b7-6fce547a63fc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:30.614125 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 784/1568] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:41:30.615477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 785/1569] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/projects => generated 375 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:30.620351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11e183-41e4-4fae-8307-ce7f13cb8aaa None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.620351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11e183-41e4-4fae-8307-ce7f13cb8aaa None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.620351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11e183-41e4-4fae-8307-ce7f13cb8aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.620631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7f816-ffdb-4c4d-b362-e43cba77277e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.635977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9d11e183-41e4-4fae-8307-ce7f13cb8aaa None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:30.637226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9d11e183-41e4-4fae-8307-ce7f13cb8aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:30.683175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7f816-ffdb-4c4d-b362-e43cba77277e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.687935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5fc78-267f-472d-a8b7-8599e2c6895d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.688610 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5fc78-267f-472d-a8b7-8599e2c6895d admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.688897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5fc78-267f-472d-a8b7-8599e2c6895d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.689263 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5fc78-267f-472d-a8b7-8599e2c6895d admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.689955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5fc78-267f-472d-a8b7-8599e2c6895d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.692028 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5fc78-267f-472d-a8b7-8599e2c6895d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:30.702094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 785/1570] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:30.706353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11e183-41e4-4fae-8307-ce7f13cb8aaa None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.712799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11e183-41e4-4fae-8307-ce7f13cb8aaa None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.717476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a62fd6-b193-4341-875e-fcf87528ed89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.718743 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a62fd6-b193-4341-875e-fcf87528ed89 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.719035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a62fd6-b193-4341-875e-fcf87528ed89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.719314 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a62fd6-b193-4341-875e-fcf87528ed89 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.720003 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a62fd6-b193-4341-875e-fcf87528ed89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.721017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-e1a62fd6-b193-4341-875e-fcf87528ed89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:41:30.721017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: warnings.warn(msg) May 23 15:41:30.721017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: May 23 15:41:30.722265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a62fd6-b193-4341-875e-fcf87528ed89 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:30.809988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 786/1571] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:30.809988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5fc78-267f-472d-a8b7-8599e2c6895d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.818132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5fc78-267f-472d-a8b7-8599e2c6895d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.823060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2924b-6bd3-4e67-8a99-64b0fda67e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.823744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2924b-6bd3-4e67-8a99-64b0fda67e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.824034 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2924b-6bd3-4e67-8a99-64b0fda67e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.824319 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2924b-6bd3-4e67-8a99-64b0fda67e9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.825112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2924b-6bd3-4e67-8a99-64b0fda67e9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:41:30.825414 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2924b-6bd3-4e67-8a99-64b0fda67e9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.827063 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2924b-6bd3-4e67-8a99-64b0fda67e9d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:30.840222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 786/1572] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:30.840222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a62fd6-b193-4341-875e-fcf87528ed89 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.841088 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 787/1573] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:41:30.841990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2924b-6bd3-4e67-8a99-64b0fda67e9d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.852601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2924b-6bd3-4e67-8a99-64b0fda67e9d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.854475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a62fd6-b193-4341-875e-fcf87528ed89 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.857259 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f7d24-3b62-4280-a963-cd0f3f0a8eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.857936 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f7d24-3b62-4280-a963-cd0f3f0a8eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.858227 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f7d24-3b62-4280-a963-cd0f3f0a8eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.859104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f7d24-3b62-4280-a963-cd0f3f0a8eb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.859809 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f7d24-3b62-4280-a963-cd0f3f0a8eb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.860843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0fe85e-c6de-451a-b863-e8da7c2a30a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.861545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fe85e-c6de-451a-b863-e8da7c2a30a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.861831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fe85e-c6de-451a-b863-e8da7c2a30a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.861941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f7d24-3b62-4280-a963-cd0f3f0a8eb8 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:30.862113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fe85e-c6de-451a-b863-e8da7c2a30a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.862872 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0fe85e-c6de-451a-b863-e8da7c2a30a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:41:30.863316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0fe85e-c6de-451a-b863-e8da7c2a30a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.864935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0fe85e-c6de-451a-b863-e8da7c2a30a6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:30.881779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 787/1574] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:41:30.881779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0fe85e-c6de-451a-b863-e8da7c2a30a6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.893413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0fe85e-c6de-451a-b863-e8da7c2a30a6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.898204 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b9252-50c4-46f7-9ca5-bc31a8364de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.898869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b9252-50c4-46f7-9ca5-bc31a8364de7 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.899158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b9252-50c4-46f7-9ca5-bc31a8364de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.899443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b9252-50c4-46f7-9ca5-bc31a8364de7 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.900122 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b9252-50c4-46f7-9ca5-bc31a8364de7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.902109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b9252-50c4-46f7-9ca5-bc31a8364de7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:30.956540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 788/1575] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:30.957451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f7d24-3b62-4280-a963-cd0f3f0a8eb8 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:30.966708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f7d24-3b62-4280-a963-cd0f3f0a8eb8 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:30.971796 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c9c4d-1914-498d-8ebc-c930b967944f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:30.972422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c9c4d-1914-498d-8ebc-c930b967944f admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:30.972706 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c9c4d-1914-498d-8ebc-c930b967944f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:30.972985 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c9c4d-1914-498d-8ebc-c930b967944f admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:30.973690 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c9c4d-1914-498d-8ebc-c930b967944f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:30.975594 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c9c4d-1914-498d-8ebc-c930b967944f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.014451 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 788/1576] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:31.014451 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b9252-50c4-46f7-9ca5-bc31a8364de7 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.024124 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b9252-50c4-46f7-9ca5-bc31a8364de7 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.028725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae57e30-42df-47d8-8242-45387f4f58dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.029371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae57e30-42df-47d8-8242-45387f4f58dd admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.029662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae57e30-42df-47d8-8242-45387f4f58dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.029944 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae57e30-42df-47d8-8242-45387f4f58dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.031299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae57e30-42df-47d8-8242-45387f4f58dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:41:31.031613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae57e30-42df-47d8-8242-45387f4f58dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.033238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae57e30-42df-47d8-8242-45387f4f58dd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.045258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 789/1577] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:41:31.046706 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae57e30-42df-47d8-8242-45387f4f58dd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.056279 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae57e30-42df-47d8-8242-45387f4f58dd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.061023 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c4780-be73-4705-a592-d90dd84c895a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.061744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c4780-be73-4705-a592-d90dd84c895a admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.062143 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c4780-be73-4705-a592-d90dd84c895a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.062431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c4780-be73-4705-a592-d90dd84c895a admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.063165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c4780-be73-4705-a592-d90dd84c895a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.064725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c4780-be73-4705-a592-d90dd84c895a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.074956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 790/1578] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:31.075757 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c4780-be73-4705-a592-d90dd84c895a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.085569 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c4780-be73-4705-a592-d90dd84c895a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.088786 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 789/1579] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:30 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:31.089571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c9c4d-1914-498d-8ebc-c930b967944f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.092258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c443a68-f888-4696-ad7e-347329667e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.092258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c443a68-f888-4696-ad7e-347329667e39 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.092258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c443a68-f888-4696-ad7e-347329667e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.092258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c443a68-f888-4696-ad7e-347329667e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.092622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c443a68-f888-4696-ad7e-347329667e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.094375 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c443a68-f888-4696-ad7e-347329667e39 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.104277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 791/1580] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:31.105052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c443a68-f888-4696-ad7e-347329667e39 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.114532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c443a68-f888-4696-ad7e-347329667e39 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.151454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c9c4d-1914-498d-8ebc-c930b967944f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.156826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a174f-5b5d-4f8e-89e7-9447da6014ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.157653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a174f-5b5d-4f8e-89e7-9447da6014ba admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.157951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a174f-5b5d-4f8e-89e7-9447da6014ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.158254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a174f-5b5d-4f8e-89e7-9447da6014ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.158998 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a174f-5b5d-4f8e-89e7-9447da6014ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.160954 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a174f-5b5d-4f8e-89e7-9447da6014ba admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.173387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17ae03-d629-4c84-adf0-65646f2e7e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.174056 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17ae03-d629-4c84-adf0-65646f2e7e21 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.174344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17ae03-d629-4c84-adf0-65646f2e7e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.174630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17ae03-d629-4c84-adf0-65646f2e7e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.175353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17ae03-d629-4c84-adf0-65646f2e7e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.177320 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17ae03-d629-4c84-adf0-65646f2e7e21 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.181461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:31.231777 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 790/1581] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:31 2022] POST /identity/v3/projects => generated 405 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:31.232674 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a174f-5b5d-4f8e-89e7-9447da6014ba admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.245767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:31.270320 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 792/1582] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:31 2022] POST /identity/v3/projects => generated 393 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:31.270320 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17ae03-d629-4c84-adf0-65646f2e7e21 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.295699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a174f-5b5d-4f8e-89e7-9447da6014ba admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.329644 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17ae03-d629-4c84-adf0-65646f2e7e21 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.334323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185d18f-ee7d-4cec-a6cd-c50c100fc7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.335000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185d18f-ee7d-4cec-a6cd-c50c100fc7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.335290 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185d18f-ee7d-4cec-a6cd-c50c100fc7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.335568 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185d18f-ee7d-4cec-a6cd-c50c100fc7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.336319 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185d18f-ee7d-4cec-a6cd-c50c100fc7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.337954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185d18f-ee7d-4cec-a6cd-c50c100fc7d9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.348996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 793/1583] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:31.349864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185d18f-ee7d-4cec-a6cd-c50c100fc7d9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.351134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6d35f-451a-4ff8-9329-e3c9852b2dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.351810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6d35f-451a-4ff8-9329-e3c9852b2dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.352095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6d35f-451a-4ff8-9329-e3c9852b2dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.352374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6d35f-451a-4ff8-9329-e3c9852b2dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.353125 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6d35f-451a-4ff8-9329-e3c9852b2dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.354871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6d35f-451a-4ff8-9329-e3c9852b2dc1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.360184 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185d18f-ee7d-4cec-a6cd-c50c100fc7d9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.365430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be98be-f65e-404b-84a5-274bbb2219aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.370603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 791/1584] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:31.370725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be98be-f65e-404b-84a5-274bbb2219aa admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.370725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be98be-f65e-404b-84a5-274bbb2219aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.370725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be98be-f65e-404b-84a5-274bbb2219aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.370950 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6d35f-451a-4ff8-9329-e3c9852b2dc1 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.371036 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be98be-f65e-404b-84a5-274bbb2219aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.371036 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be98be-f65e-404b-84a5-274bbb2219aa admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.376401 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6d35f-451a-4ff8-9329-e3c9852b2dc1 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.380975 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045b17d-0a17-44e7-b6f3-73b9d7aae919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.381646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045b17d-0a17-44e7-b6f3-73b9d7aae919 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.381941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045b17d-0a17-44e7-b6f3-73b9d7aae919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.382208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045b17d-0a17-44e7-b6f3-73b9d7aae919 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.382946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045b17d-0a17-44e7-b6f3-73b9d7aae919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.384561 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045b17d-0a17-44e7-b6f3-73b9d7aae919 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.395579 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 792/1585] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:31.396409 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045b17d-0a17-44e7-b6f3-73b9d7aae919 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.405934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045b17d-0a17-44e7-b6f3-73b9d7aae919 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.410393 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cd929-6d2d-40dd-9b6b-db3a3461fb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.411013 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cd929-6d2d-40dd-9b6b-db3a3461fb54 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.411300 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cd929-6d2d-40dd-9b6b-db3a3461fb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.411575 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cd929-6d2d-40dd-9b6b-db3a3461fb54 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.412235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cd929-6d2d-40dd-9b6b-db3a3461fb54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.414123 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cd929-6d2d-40dd-9b6b-db3a3461fb54 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.449080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 794/1586] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:31 2022] POST /identity/v3/projects => generated 405 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:31.450016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be98be-f65e-404b-84a5-274bbb2219aa admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.510837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be98be-f65e-404b-84a5-274bbb2219aa admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.516243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3d6da-1471-425a-bae2-470ee4590c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.516919 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3d6da-1471-425a-bae2-470ee4590c7e admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.517236 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3d6da-1471-425a-bae2-470ee4590c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.517522 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3d6da-1471-425a-bae2-470ee4590c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.518244 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3d6da-1471-425a-bae2-470ee4590c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.519872 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3d6da-1471-425a-bae2-470ee4590c7e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.529731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 795/1587] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:31.530540 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3d6da-1471-425a-bae2-470ee4590c7e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.542060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 793/1588] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:31 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:31.542263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3d6da-1471-425a-bae2-470ee4590c7e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.542380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cd929-6d2d-40dd-9b6b-db3a3461fb54 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.544413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c9aec-9d63-475e-9a6c-47af486225f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.545283 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c9aec-9d63-475e-9a6c-47af486225f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.545579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c9aec-9d63-475e-9a6c-47af486225f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.545899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c9aec-9d63-475e-9a6c-47af486225f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.546765 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c9aec-9d63-475e-9a6c-47af486225f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.548650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c9aec-9d63-475e-9a6c-47af486225f0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.550220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cd929-6d2d-40dd-9b6b-db3a3461fb54 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.554939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db20e20-36fc-4c97-ab9f-d739816e0f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.560634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db20e20-36fc-4c97-ab9f-d739816e0f4d admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.560634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db20e20-36fc-4c97-ab9f-d739816e0f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.560634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db20e20-36fc-4c97-ab9f-d739816e0f4d admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.560634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db20e20-36fc-4c97-ab9f-d739816e0f4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.560634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db20e20-36fc-4c97-ab9f-d739816e0f4d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.652756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 796/1589] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:31 2022] POST /identity/v3/users => generated 303 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:31.653742 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c9aec-9d63-475e-9a6c-47af486225f0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.663003 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c9aec-9d63-475e-9a6c-47af486225f0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.667470 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c36bb-f44e-46f7-be74-8f5d0b6d0d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.668360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c36bb-f44e-46f7-be74-8f5d0b6d0d67 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.668649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c36bb-f44e-46f7-be74-8f5d0b6d0d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.668927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c36bb-f44e-46f7-be74-8f5d0b6d0d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.669678 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c36bb-f44e-46f7-be74-8f5d0b6d0d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.671283 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c36bb-f44e-46f7-be74-8f5d0b6d0d67 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.681984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 797/1590] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:31.682876 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c36bb-f44e-46f7-be74-8f5d0b6d0d67 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.685244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 794/1591] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:31 2022] POST /identity/v3/users => generated 297 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:31.686139 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db20e20-36fc-4c97-ab9f-d739816e0f4d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.693421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c36bb-f44e-46f7-be74-8f5d0b6d0d67 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.696688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db20e20-36fc-4c97-ab9f-d739816e0f4d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.701495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0f61a-e02d-4171-9313-85c0bd7ad908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.702776 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0f61a-e02d-4171-9313-85c0bd7ad908 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.703141 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0f61a-e02d-4171-9313-85c0bd7ad908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.703395 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0f61a-e02d-4171-9313-85c0bd7ad908 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.704114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0f61a-e02d-4171-9313-85c0bd7ad908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.705771 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0f61a-e02d-4171-9313-85c0bd7ad908 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.706731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c670cf-3b36-49aa-b53e-1c2232739c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.707613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c670cf-3b36-49aa-b53e-1c2232739c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.707927 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c670cf-3b36-49aa-b53e-1c2232739c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.708200 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c670cf-3b36-49aa-b53e-1c2232739c1b admin admin] PATH_INFO: `/v3/projects/f380b8a58cfc4b71b86681885ed58031/users/48352fabb230456690e795d91115605b/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.716208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 798/1592] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:31.717424 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0f61a-e02d-4171-9313-85c0bd7ad908 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.724933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c670cf-3b36-49aa-b53e-1c2232739c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f380b8a58cfc4b71b86681885ed58031, user_id=48352fabb230456690e795d91115605b, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.726068 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: WARNING py.warnings [None req-40c670cf-3b36-49aa-b53e-1c2232739c1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:41:31.726068 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: warnings.warn(msg) May 23 15:41:31.726068 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: May 23 15:41:31.727501 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0f61a-e02d-4171-9313-85c0bd7ad908 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.728044 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c670cf-3b36-49aa-b53e-1c2232739c1b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.732131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f87b1a-bb4a-47f9-ac88-b76a1cb9c5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.732812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f87b1a-bb4a-47f9-ac88-b76a1cb9c5c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.733128 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f87b1a-bb4a-47f9-ac88-b76a1cb9c5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.733432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f87b1a-bb4a-47f9-ac88-b76a1cb9c5c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.734112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f87b1a-bb4a-47f9-ac88-b76a1cb9c5c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.735996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f87b1a-bb4a-47f9-ac88-b76a1cb9c5c1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.784419 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 795/1593] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:31 2022] PUT /identity/v3/projects/f380b8a58cfc4b71b86681885ed58031/users/48352fabb230456690e795d91115605b/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:31.785313 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c670cf-3b36-49aa-b53e-1c2232739c1b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.803393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:31.812195 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 799/1594] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:31 2022] POST /identity/v3/users => generated 303 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:31.812195 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f87b1a-bb4a-47f9-ac88-b76a1cb9c5c1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.820160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f87b1a-bb4a-47f9-ac88-b76a1cb9c5c1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.835315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:31.849374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c670cf-3b36-49aa-b53e-1c2232739c1b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.854207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1e340-ebe1-4bfa-b3fe-3cc0703481b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.855124 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1e340-ebe1-4bfa-b3fe-3cc0703481b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.855409 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1e340-ebe1-4bfa-b3fe-3cc0703481b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.855685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1e340-ebe1-4bfa-b3fe-3cc0703481b3 admin admin] PATH_INFO: `/v3/projects/e3a7e6bce7fc48f996703cb5eb3533fb/users/49f723f9ef414cd79a6167dcd852ffe1/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.871206 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb1e340-ebe1-4bfa-b3fe-3cc0703481b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a7e6bce7fc48f996703cb5eb3533fb, user_id=49f723f9ef414cd79a6167dcd852ffe1, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.873974 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb1e340-ebe1-4bfa-b3fe-3cc0703481b3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.876185 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a864b-8b50-4ea0-9dd9-0909cc7c8fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.876855 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a864b-8b50-4ea0-9dd9-0909cc7c8fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.877173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a864b-8b50-4ea0-9dd9-0909cc7c8fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.877872 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a864b-8b50-4ea0-9dd9-0909cc7c8fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.878173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a864b-8b50-4ea0-9dd9-0909cc7c8fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.879746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a864b-8b50-4ea0-9dd9-0909cc7c8fb0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.889369 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 800/1595] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:31.890183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a864b-8b50-4ea0-9dd9-0909cc7c8fb0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.899535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a864b-8b50-4ea0-9dd9-0909cc7c8fb0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.904117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e74ead-7cdd-4ac4-8ab4-f7a721d1d8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.904678 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e74ead-7cdd-4ac4-8ab4-f7a721d1d8bd admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.904960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e74ead-7cdd-4ac4-8ab4-f7a721d1d8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.905301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e74ead-7cdd-4ac4-8ab4-f7a721d1d8bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.906135 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e74ead-7cdd-4ac4-8ab4-f7a721d1d8bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.907711 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e74ead-7cdd-4ac4-8ab4-f7a721d1d8bd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:31.917073 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 801/1596] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:31.918103 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e74ead-7cdd-4ac4-8ab4-f7a721d1d8bd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.927394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e74ead-7cdd-4ac4-8ab4-f7a721d1d8bd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:31.931811 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0b030-9776-4493-b413-3e7934493882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:31.932638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0b030-9776-4493-b413-3e7934493882 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:31.932919 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0b030-9776-4493-b413-3e7934493882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:31.933227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0b030-9776-4493-b413-3e7934493882 admin admin] PATH_INFO: `/v3/projects/8d7ac236cd4d414fb70eb8da6da53484/users/5236a93fc3a84d27bf3d150e85eb2589/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:31.939712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 796/1597] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:31 2022] PUT /identity/v3/projects/e3a7e6bce7fc48f996703cb5eb3533fb/users/49f723f9ef414cd79a6167dcd852ffe1/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:31.940546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1e340-ebe1-4bfa-b3fe-3cc0703481b3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:31.956801 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0b030-9776-4493-b413-3e7934493882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d7ac236cd4d414fb70eb8da6da53484, user_id=5236a93fc3a84d27bf3d150e85eb2589, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:31.960566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0b030-9776-4493-b413-3e7934493882 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:32.000359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1e340-ebe1-4bfa-b3fe-3cc0703481b3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:32.004936 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2414b4-9a6f-4543-a26d-d7adaeef768f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:32.005626 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2414b4-9a6f-4543-a26d-d7adaeef768f admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.005911 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2414b4-9a6f-4543-a26d-d7adaeef768f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.006191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2414b4-9a6f-4543-a26d-d7adaeef768f admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.006907 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2414b4-9a6f-4543-a26d-d7adaeef768f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:32.008487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2414b4-9a6f-4543-a26d-d7adaeef768f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:32.017906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 797/1598] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:32.018663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2414b4-9a6f-4543-a26d-d7adaeef768f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:32.027763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2414b4-9a6f-4543-a26d-d7adaeef768f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:32.031213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 802/1599] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:31 2022] PUT /identity/v3/projects/8d7ac236cd4d414fb70eb8da6da53484/users/5236a93fc3a84d27bf3d150e85eb2589/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:32.031213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c127cbe-07fa-4e18-a738-c175f188252c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.031213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c127cbe-07fa-4e18-a738-c175f188252c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.033004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c127cbe-07fa-4e18-a738-c175f188252c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.058725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2c127cbe-07fa-4e18-a738-c175f188252c None None] MFA Rules not processed for user `48352fabb230456690e795d91115605b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:32.062183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2c127cbe-07fa-4e18-a738-c175f188252c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:32.088316 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57261426-cec5-49d2-b98b-57aeb3d6fb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:32.089265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261426-cec5-49d2-b98b-57aeb3d6fb43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.089556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261426-cec5-49d2-b98b-57aeb3d6fb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.089841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261426-cec5-49d2-b98b-57aeb3d6fb43 admin admin] PATH_INFO: `/v3/projects/0a819820263647f784079581c7527921/users/916c7e4c45a94167ba39db8c7fec0d19/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.104606 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57261426-cec5-49d2-b98b-57aeb3d6fb43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a819820263647f784079581c7527921, user_id=916c7e4c45a94167ba39db8c7fec0d19, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:32.107331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57261426-cec5-49d2-b98b-57aeb3d6fb43 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:32.125808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 803/1600] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:32.126649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c127cbe-07fa-4e18-a738-c175f188252c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:32.136742 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c127cbe-07fa-4e18-a738-c175f188252c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:32.142061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52ba33-0ce8-4118-8984-3a9f8cd8285b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:32.142721 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52ba33-0ce8-4118-8984-3a9f8cd8285b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.143006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52ba33-0ce8-4118-8984-3a9f8cd8285b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.143284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52ba33-0ce8-4118-8984-3a9f8cd8285b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.143999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52ba33-0ce8-4118-8984-3a9f8cd8285b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:32.145617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52ba33-0ce8-4118-8984-3a9f8cd8285b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:32.156555 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 804/1601] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:32.158036 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52ba33-0ce8-4118-8984-3a9f8cd8285b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:32.168265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52ba33-0ce8-4118-8984-3a9f8cd8285b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:32.173625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450fffb-3512-44c8-9f31-2fd427ca1348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:32.176909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450fffb-3512-44c8-9f31-2fd427ca1348 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.176909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450fffb-3512-44c8-9f31-2fd427ca1348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.176909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450fffb-3512-44c8-9f31-2fd427ca1348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.176909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450fffb-3512-44c8-9f31-2fd427ca1348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:32.177289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450fffb-3512-44c8-9f31-2fd427ca1348 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:32.179739 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 798/1602] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:32 2022] PUT /identity/v3/projects/0a819820263647f784079581c7527921/users/916c7e4c45a94167ba39db8c7fec0d19/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:32.180655 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57261426-cec5-49d2-b98b-57aeb3d6fb43 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:32.187413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 805/1603] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:32.188671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b45fa-7f8a-4ea3-9139-50f304f3b81c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.188954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b45fa-7f8a-4ea3-9139-50f304f3b81c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.189258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b45fa-7f8a-4ea3-9139-50f304f3b81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.214816 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-758b45fa-7f8a-4ea3-9139-50f304f3b81c None None] MFA Rules not processed for user `49f723f9ef414cd79a6167dcd852ffe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:32.216088 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-758b45fa-7f8a-4ea3-9139-50f304f3b81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:32.234849 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:32.236586 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:32.249715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57261426-cec5-49d2-b98b-57aeb3d6fb43 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:32.254437 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914e274-e32f-4941-bd4e-e131eca9aede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:32.255116 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914e274-e32f-4941-bd4e-e131eca9aede admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.255410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914e274-e32f-4941-bd4e-e131eca9aede admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.255700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914e274-e32f-4941-bd4e-e131eca9aede admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.256431 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914e274-e32f-4941-bd4e-e131eca9aede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:32.258115 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914e274-e32f-4941-bd4e-e131eca9aede admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:32.268024 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 799/1604] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:32.270558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914e274-e32f-4941-bd4e-e131eca9aede admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:32.278727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914e274-e32f-4941-bd4e-e131eca9aede admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:32.283957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea354143-a34e-4ad9-a3c7-92795a736a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:32.284510 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 806/1605] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:32.285209 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea354143-a34e-4ad9-a3c7-92795a736a5c admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.285539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea354143-a34e-4ad9-a3c7-92795a736a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.285973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb2bd6-448f-4c3d-baa3-af14d26340a6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.286099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea354143-a34e-4ad9-a3c7-92795a736a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.286284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb2bd6-448f-4c3d-baa3-af14d26340a6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.286554 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb2bd6-448f-4c3d-baa3-af14d26340a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.286736 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea354143-a34e-4ad9-a3c7-92795a736a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:32.288798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea354143-a34e-4ad9-a3c7-92795a736a5c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:32.301034 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 800/1606] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:32.302579 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d29abc-5114-4940-928c-6b360d10198c None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.302698 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d29abc-5114-4940-928c-6b360d10198c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.302955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d29abc-5114-4940-928c-6b360d10198c None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.304670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 801/1607] 10.222.0.64 () {54 vars in 961 bytes} [Mon May 23 15:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:41:32.305425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d29abc-5114-4940-928c-6b360d10198c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:32.310297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-17cb2bd6-448f-4c3d-baa3-af14d26340a6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:32.311637 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-17cb2bd6-448f-4c3d-baa3-af14d26340a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:32.318012 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d29abc-5114-4940-928c-6b360d10198c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:32.319676 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 807/1608] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:32.320916 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b3271-30b4-450f-b5ad-219cd20d0673 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.321247 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b3271-30b4-450f-b5ad-219cd20d0673 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.321533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b3271-30b4-450f-b5ad-219cd20d0673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.326790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaba2ca-b3f2-4c0b-a5a4-effea1aa9ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:32.327472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaba2ca-b3f2-4c0b-a5a4-effea1aa9ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.327761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaba2ca-b3f2-4c0b-a5a4-effea1aa9ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.328039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaba2ca-b3f2-4c0b-a5a4-effea1aa9ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.328765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaba2ca-b3f2-4c0b-a5a4-effea1aa9ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:32.330429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaba2ca-b3f2-4c0b-a5a4-effea1aa9ea3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:32.340896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a31b3271-30b4-450f-b5ad-219cd20d0673 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:32.345670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 802/1609] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:32.345670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaba2ca-b3f2-4c0b-a5a4-effea1aa9ea3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:32.345885 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a31b3271-30b4-450f-b5ad-219cd20d0673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:32.353319 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 808/1610] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:32.354294 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b3271-30b4-450f-b5ad-219cd20d0673 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:32.354480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaba2ca-b3f2-4c0b-a5a4-effea1aa9ea3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:32.359776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dfc1ec-b1c3-4a47-add7-a727d090750f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:32.362520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfc1ec-b1c3-4a47-add7-a727d090750f admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.362520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfc1ec-b1c3-4a47-add7-a727d090750f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.362520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfc1ec-b1c3-4a47-add7-a727d090750f admin admin] PATH_INFO: `/v3/projects/ca944beed5584d4f816e39cb1c6098d9/users/1bea05d85a4f4b0e9a0e730d8fd8c7a8/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.365379 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b3271-30b4-450f-b5ad-219cd20d0673 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:32.370942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839f9a4-7b89-49fc-9799-0685a56ca771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:32.371832 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839f9a4-7b89-49fc-9799-0685a56ca771 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.372114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839f9a4-7b89-49fc-9799-0685a56ca771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.372390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839f9a4-7b89-49fc-9799-0685a56ca771 admin admin] PATH_INFO: `/v3/projects/05c33a8edc9e4b34ad4d7e6da04fa44a/users/60cd50c000ed4dbb8eca4a2900bf43a1/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.377204 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dfc1ec-b1c3-4a47-add7-a727d090750f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca944beed5584d4f816e39cb1c6098d9, user_id=1bea05d85a4f4b0e9a0e730d8fd8c7a8, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:32.379876 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dfc1ec-b1c3-4a47-add7-a727d090750f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:32.386995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839f9a4-7b89-49fc-9799-0685a56ca771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05c33a8edc9e4b34ad4d7e6da04fa44a, user_id=60cd50c000ed4dbb8eca4a2900bf43a1, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:32.389702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839f9a4-7b89-49fc-9799-0685a56ca771 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:32.470513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 809/1611] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:32 2022] PUT /identity/v3/projects/05c33a8edc9e4b34ad4d7e6da04fa44a/users/60cd50c000ed4dbb8eca4a2900bf43a1/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:32.470850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 803/1612] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:32 2022] PUT /identity/v3/projects/ca944beed5584d4f816e39cb1c6098d9/users/1bea05d85a4f4b0e9a0e730d8fd8c7a8/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:32.471254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839f9a4-7b89-49fc-9799-0685a56ca771 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:32.472285 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37df09-19d7-4900-9141-d1f40324db4a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.472580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37df09-19d7-4900-9141-d1f40324db4a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.472860 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37df09-19d7-4900-9141-d1f40324db4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.493776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8a37df09-19d7-4900-9141-d1f40324db4a None None] MFA Rules not processed for user `8da96bed73794fee8f719ec904936164`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:32.495104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8a37df09-19d7-4900-9141-d1f40324db4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:32.533051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839f9a4-7b89-49fc-9799-0685a56ca771 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:32.563236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 804/1613] 10.222.0.64 () {58 vars in 1058 bytes} [Mon May 23 15:41:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:32.566162 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997f0ff-0c4a-4213-8969-856aae783343 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.566162 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997f0ff-0c4a-4213-8969-856aae783343 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.566162 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997f0ff-0c4a-4213-8969-856aae783343 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.567011 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 805/1614] 10.222.0.64 () {54 vars in 961 bytes} [Mon May 23 15:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:41:32.567891 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997f0ff-0c4a-4213-8969-856aae783343 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:32.587709 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f8fef-a748-43bb-bdad-6fd538057100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:32.588607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f8fef-a748-43bb-bdad-6fd538057100 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.589147 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f8fef-a748-43bb-bdad-6fd538057100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.589266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f8fef-a748-43bb-bdad-6fd538057100 admin admin] PATH_INFO: `/v3/projects/65483f2a0c914022b38e501691095c51/users/a2f1d435732b4ff0a60599309c66eeb5/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.604218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f8fef-a748-43bb-bdad-6fd538057100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65483f2a0c914022b38e501691095c51, user_id=a2f1d435732b4ff0a60599309c66eeb5, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:32.606990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f8fef-a748-43bb-bdad-6fd538057100 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:32.624242 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997f0ff-0c4a-4213-8969-856aae783343 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:32.628001 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccb367-7920-4222-b190-b3377690518e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:32.628678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccb367-7920-4222-b190-b3377690518e service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.628960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccb367-7920-4222-b190-b3377690518e service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.629264 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccb367-7920-4222-b190-b3377690518e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.638094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccb367-7920-4222-b190-b3377690518e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:32.640211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccb367-7920-4222-b190-b3377690518e service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:32.649608 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 806/1615] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:32.652268 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccb367-7920-4222-b190-b3377690518e service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:32.659790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccb367-7920-4222-b190-b3377690518e service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:32.665105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f78c2-564a-4983-9120-0ffbc5946293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:32.666222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f78c2-564a-4983-9120-0ffbc5946293 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.666339 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f78c2-564a-4983-9120-0ffbc5946293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.666619 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f78c2-564a-4983-9120-0ffbc5946293 admin admin] PATH_INFO: `/v3/projects/bcd7eb803f544d04bb7a1eca4b7ad913/users/f109fbb1f8d14ffdb6ab4efe06ad9fac/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.673167 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 810/1616] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:32 2022] PUT /identity/v3/projects/65483f2a0c914022b38e501691095c51/users/a2f1d435732b4ff0a60599309c66eeb5/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:32.674119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f8fef-a748-43bb-bdad-6fd538057100 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:32.686608 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:32.687671 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f78c2-564a-4983-9120-0ffbc5946293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcd7eb803f544d04bb7a1eca4b7ad913, user_id=f109fbb1f8d14ffdb6ab4efe06ad9fac, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:32.690436 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f78c2-564a-4983-9120-0ffbc5946293 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:32.730334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f8fef-a748-43bb-bdad-6fd538057100 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:32.734184 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c0945-3df9-4d8b-9c72-e15aa0ac801c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:32.738538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c0945-3df9-4d8b-9c72-e15aa0ac801c service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.738538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c0945-3df9-4d8b-9c72-e15aa0ac801c service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.738538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c0945-3df9-4d8b-9c72-e15aa0ac801c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.761389 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 807/1617] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:32 2022] PUT /identity/v3/projects/bcd7eb803f544d04bb7a1eca4b7ad913/users/f109fbb1f8d14ffdb6ab4efe06ad9fac/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:32.767067 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40678a5-220b-4a2b-8c65-102bdd0e9a47 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.767454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40678a5-220b-4a2b-8c65-102bdd0e9a47 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.767981 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40678a5-220b-4a2b-8c65-102bdd0e9a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.788080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:32.797167 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e40678a5-220b-4a2b-8c65-102bdd0e9a47 None None] MFA Rules not processed for user `1bea05d85a4f4b0e9a0e730d8fd8c7a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:32.798530 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e40678a5-220b-4a2b-8c65-102bdd0e9a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:32.799986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c0945-3df9-4d8b-9c72-e15aa0ac801c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:32.801799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c0945-3df9-4d8b-9c72-e15aa0ac801c service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:32.812464 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 811/1618] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:32.813642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c17f5a-8042-4cce-9954-c6a4138dab63 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.813939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c17f5a-8042-4cce-9954-c6a4138dab63 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.814226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c17f5a-8042-4cce-9954-c6a4138dab63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.841694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-84c17f5a-8042-4cce-9954-c6a4138dab63 None None] MFA Rules not processed for user `60cd50c000ed4dbb8eca4a2900bf43a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:32.843731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-84c17f5a-8042-4cce-9954-c6a4138dab63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:32.866740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 808/1619] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:32.868187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40678a5-220b-4a2b-8c65-102bdd0e9a47 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:32.910248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 812/1620] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:32 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:32.911540 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144663a-fe75-4f91-9e35-9df9e257021a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.911837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144663a-fe75-4f91-9e35-9df9e257021a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.912112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144663a-fe75-4f91-9e35-9df9e257021a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.933771 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8144663a-fe75-4f91-9e35-9df9e257021a None None] MFA Rules not processed for user `8da96bed73794fee8f719ec904936164`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:32.935071 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8144663a-fe75-4f91-9e35-9df9e257021a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:32.941300 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40678a5-220b-4a2b-8c65-102bdd0e9a47 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:32.947269 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 813/1621] 10.222.0.64 () {58 vars in 1058 bytes} [Mon May 23 15:41:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:32.948105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bcc53-01b5-4c60-90bf-f00d2f7a31e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:32.948771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bcc53-01b5-4c60-90bf-f00d2f7a31e9 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.949104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bcc53-01b5-4c60-90bf-f00d2f7a31e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.953127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bcc53-01b5-4c60-90bf-f00d2f7a31e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.957587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc605ec-185b-4c71-b27c-aa01789b619b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:32.957882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc605ec-185b-4c71-b27c-aa01789b619b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:32.958427 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc605ec-185b-4c71-b27c-aa01789b619b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:32.977957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9dc605ec-185b-4c71-b27c-aa01789b619b None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:32.979194 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9dc605ec-185b-4c71-b27c-aa01789b619b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:33.010898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bcc53-01b5-4c60-90bf-f00d2f7a31e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:33.012622 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bcc53-01b5-4c60-90bf-f00d2f7a31e9 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:33.023651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 809/1622] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:33.024903 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8860c56-87f2-476a-a103-4c7f7eaf9448 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:33.025402 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8860c56-87f2-476a-a103-4c7f7eaf9448 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:33.025599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8860c56-87f2-476a-a103-4c7f7eaf9448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:33.040055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:33.044985 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a8860c56-87f2-476a-a103-4c7f7eaf9448 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:33.046926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a8860c56-87f2-476a-a103-4c7f7eaf9448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:33.089960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:33.096348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 814/1623] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:33.096348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc605ec-185b-4c71-b27c-aa01789b619b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:33.119617 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 810/1624] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:33.120484 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8860c56-87f2-476a-a103-4c7f7eaf9448 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:33.123366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc605ec-185b-4c71-b27c-aa01789b619b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:33.134496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8860c56-87f2-476a-a103-4c7f7eaf9448 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:33.136304 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c40a1c-4a48-463d-b874-3b38de306fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:33.137387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c40a1c-4a48-463d-b874-3b38de306fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:33.137853 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c40a1c-4a48-463d-b874-3b38de306fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:33.138960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c40a1c-4a48-463d-b874-3b38de306fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:33.141073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b8abd-745b-43b3-8f51-91a63fac6177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:33.141232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c40a1c-4a48-463d-b874-3b38de306fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:33.141912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b8abd-745b-43b3-8f51-91a63fac6177 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:33.142270 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b8abd-745b-43b3-8f51-91a63fac6177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:33.142621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b8abd-745b-43b3-8f51-91a63fac6177 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:33.143442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b8abd-745b-43b3-8f51-91a63fac6177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:33.145477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b8abd-745b-43b3-8f51-91a63fac6177 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:33.147157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c40a1c-4a48-463d-b874-3b38de306fe5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:33.222982 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 811/1625] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:33 2022] POST /identity/v3/projects => generated 403 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:33.223949 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b8abd-745b-43b3-8f51-91a63fac6177 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:33.251572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 815/1626] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:33 2022] POST /identity/v3/projects => generated 377 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:33.252862 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c40a1c-4a48-463d-b874-3b38de306fe5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:33.256627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c9c40a1c-4a48-463d-b874-3b38de306fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:33.332754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b8abd-745b-43b3-8f51-91a63fac6177 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:33.353694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c40a1c-4a48-463d-b874-3b38de306fe5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:33.360029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95aa2f-d18e-4a7b-ba2d-d8ae49f8f1a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:33.360778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95aa2f-d18e-4a7b-ba2d-d8ae49f8f1a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:33.361217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95aa2f-d18e-4a7b-ba2d-d8ae49f8f1a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:33.361567 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95aa2f-d18e-4a7b-ba2d-d8ae49f8f1a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:33.421946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7de6d-36cc-494c-a23a-60f658d6e151 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:33.422913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7de6d-36cc-494c-a23a-60f658d6e151 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:33.423348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7de6d-36cc-494c-a23a-60f658d6e151 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:33.423778 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7de6d-36cc-494c-a23a-60f658d6e151 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:33.467255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95aa2f-d18e-4a7b-ba2d-d8ae49f8f1a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:33.469128 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95aa2f-d18e-4a7b-ba2d-d8ae49f8f1a1 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:33.479723 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 816/1627] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:33.480564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95aa2f-d18e-4a7b-ba2d-d8ae49f8f1a1 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:33.489036 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95aa2f-d18e-4a7b-ba2d-d8ae49f8f1a1 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:33.492833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9653e94-d111-4707-aed5-ae43a0661c1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:33.493511 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9653e94-d111-4707-aed5-ae43a0661c1e service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:33.493803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9653e94-d111-4707-aed5-ae43a0661c1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:33.494040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:33.494162 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9653e94-d111-4707-aed5-ae43a0661c1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:33.506443 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7de6d-36cc-494c-a23a-60f658d6e151 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:33.508310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7de6d-36cc-494c-a23a-60f658d6e151 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:33.514039 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9653e94-d111-4707-aed5-ae43a0661c1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:33.515839 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9653e94-d111-4707-aed5-ae43a0661c1e service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:33.522759 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 812/1628] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:33.524253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7de6d-36cc-494c-a23a-60f658d6e151 service nova] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:33.537285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 817/1629] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:33.537821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7de6d-36cc-494c-a23a-60f658d6e151 service nova] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:33.539259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9653e94-d111-4707-aed5-ae43a0661c1e service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:33.543081 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d4bef-5005-43cf-adc6-2104c86817d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:33.543761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d4bef-5005-43cf-adc6-2104c86817d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:33.544053 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d4bef-5005-43cf-adc6-2104c86817d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:33.544334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d4bef-5005-43cf-adc6-2104c86817d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:33.545353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d4bef-5005-43cf-adc6-2104c86817d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:33.549096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d4bef-5005-43cf-adc6-2104c86817d5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:33.552331 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9653e94-d111-4707-aed5-ae43a0661c1e service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:33.560909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4885e9-5671-419a-b23f-ba264954fa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:33.563074 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4885e9-5671-419a-b23f-ba264954fa85 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:33.563074 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4885e9-5671-419a-b23f-ba264954fa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:33.563329 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4885e9-5671-419a-b23f-ba264954fa85 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:33.563989 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4885e9-5671-419a-b23f-ba264954fa85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:33.566306 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4885e9-5671-419a-b23f-ba264954fa85 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:33.625222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 813/1630] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:33 2022] POST /identity/v3/users => generated 308 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:33.626849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda37c81-7488-46c3-ac42-43c66c50f7c9 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:33.627201 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda37c81-7488-46c3-ac42-43c66c50f7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:33.628262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda37c81-7488-46c3-ac42-43c66c50f7c9 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:33.629854 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 814/1631] 10.222.0.64 () {54 vars in 912 bytes} [Mon May 23 15:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:41:33.641309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda37c81-7488-46c3-ac42-43c66c50f7c9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:33.654182 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda37c81-7488-46c3-ac42-43c66c50f7c9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:33.659819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ad71e-81a8-4f1b-bc97-031c779c9499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:33.660594 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ad71e-81a8-4f1b-bc97-031c779c9499 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:33.660952 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ad71e-81a8-4f1b-bc97-031c779c9499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:33.661331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ad71e-81a8-4f1b-bc97-031c779c9499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:33.662222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ad71e-81a8-4f1b-bc97-031c779c9499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:33.664187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ad71e-81a8-4f1b-bc97-031c779c9499 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:33.679066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 815/1632] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:33.679066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4de0f0-9c3e-4aa8-a712-7e5320efcad1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:33.679066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4de0f0-9c3e-4aa8-a712-7e5320efcad1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:33.679066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4de0f0-9c3e-4aa8-a712-7e5320efcad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:33.683764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 818/1633] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:33 2022] POST /identity/v3/users => generated 295 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:33.684734 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4885e9-5671-419a-b23f-ba264954fa85 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:33.704322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4885e9-5671-419a-b23f-ba264954fa85 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:33.707828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-5e4de0f0-9c3e-4aa8-a712-7e5320efcad1 None None] MFA Rules not processed for user `8da96bed73794fee8f719ec904936164`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:33.710359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5e4de0f0-9c3e-4aa8-a712-7e5320efcad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:33.710754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f443dd-4a39-4504-96b4-3356352d7cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:33.712292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f443dd-4a39-4504-96b4-3356352d7cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:33.713424 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f443dd-4a39-4504-96b4-3356352d7cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:33.713858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f443dd-4a39-4504-96b4-3356352d7cb7 admin admin] PATH_INFO: `/v3/projects/d7b18cbe6ebf4707b96c53fc3e7a7c3a/users/25d23dd65cfa4ef4a48f89beb87b51a9/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:33.732282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 816/1634] 10.222.0.64 () {58 vars in 1009 bytes} [Mon May 23 15:41:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:33.742305 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f443dd-4a39-4504-96b4-3356352d7cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7b18cbe6ebf4707b96c53fc3e7a7c3a, user_id=25d23dd65cfa4ef4a48f89beb87b51a9, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:33.745581 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f443dd-4a39-4504-96b4-3356352d7cb7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:33.748762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4de0f0-9c3e-4aa8-a712-7e5320efcad1 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:33.764729 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4de0f0-9c3e-4aa8-a712-7e5320efcad1 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:33.769786 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c39e49-b75c-43b4-bc85-a6398a862b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:33.771056 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c39e49-b75c-43b4-bc85-a6398a862b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:33.771598 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c39e49-b75c-43b4-bc85-a6398a862b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:33.772416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c39e49-b75c-43b4-bc85-a6398a862b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:33.773505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c39e49-b75c-43b4-bc85-a6398a862b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:33.775524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c39e49-b75c-43b4-bc85-a6398a862b36 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:33.788722 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 817/1635] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:33.790026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c39e49-b75c-43b4-bc85-a6398a862b36 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:33.821308 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 819/1636] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:33 2022] PUT /identity/v3/projects/d7b18cbe6ebf4707b96c53fc3e7a7c3a/users/25d23dd65cfa4ef4a48f89beb87b51a9/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:33.821308 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f443dd-4a39-4504-96b4-3356352d7cb7 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:33.861012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:33.907026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c39e49-b75c-43b4-bc85-a6398a862b36 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:33.926912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f443dd-4a39-4504-96b4-3356352d7cb7 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:33.932748 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79642e15-71cc-4d7c-a81e-51aac34b7584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:33.933687 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79642e15-71cc-4d7c-a81e-51aac34b7584 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:33.942050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79642e15-71cc-4d7c-a81e-51aac34b7584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:33.942050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79642e15-71cc-4d7c-a81e-51aac34b7584 admin admin] PATH_INFO: `/v3/projects/0e18c4b9981342249f01c73ccd2f287f/users/d430d9949e1b446ba54c490ad445855b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:33.950713 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79642e15-71cc-4d7c-a81e-51aac34b7584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e18c4b9981342249f01c73ccd2f287f, user_id=d430d9949e1b446ba54c490ad445855b, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:33.954201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79642e15-71cc-4d7c-a81e-51aac34b7584 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:33.968553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6346b36-2111-4201-8f1c-29b88bfd8207 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:33.969512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6346b36-2111-4201-8f1c-29b88bfd8207 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:33.969880 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6346b36-2111-4201-8f1c-29b88bfd8207 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:33.970303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6346b36-2111-4201-8f1c-29b88bfd8207 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:34.043916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6346b36-2111-4201-8f1c-29b88bfd8207 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:34.045522 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6346b36-2111-4201-8f1c-29b88bfd8207 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:34.053983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:34.060779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 820/1637] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:33 2022] PUT /identity/v3/projects/0e18c4b9981342249f01c73ccd2f287f/users/d430d9949e1b446ba54c490ad445855b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:34.068246 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79642e15-71cc-4d7c-a81e-51aac34b7584 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:34.073121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 818/1638] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:34.075638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d8ddb-57af-4812-87e9-a964201acecc None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:34.075935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d8ddb-57af-4812-87e9-a964201acecc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:34.076216 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d8ddb-57af-4812-87e9-a964201acecc None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:34.080095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 819/1639] 10.222.0.64 () {54 vars in 912 bytes} [Mon May 23 15:41:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:41:34.082158 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d8ddb-57af-4812-87e9-a964201acecc None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:34.161338 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79642e15-71cc-4d7c-a81e-51aac34b7584 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:34.171275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bf112-962c-4598-839a-e9686449b3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:34.172366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bf112-962c-4598-839a-e9686449b3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:34.172982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bf112-962c-4598-839a-e9686449b3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:34.178414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bf112-962c-4598-839a-e9686449b3b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:34.178414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9bf112-962c-4598-839a-e9686449b3b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:34.178414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9bf112-962c-4598-839a-e9686449b3b2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:34.193003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d8ddb-57af-4812-87e9-a964201acecc None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:34.196321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 821/1640] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:34.196321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cb763-e759-48b6-aa68-6f1676d93f1e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:34.196321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cb763-e759-48b6-aa68-6f1676d93f1e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:34.196321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cb763-e759-48b6-aa68-6f1676d93f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:34.199989 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3eda0f-da03-4f48-9281-8462de1e2a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:34.201083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3eda0f-da03-4f48-9281-8462de1e2a8c admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:34.201383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3eda0f-da03-4f48-9281-8462de1e2a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:34.201665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3eda0f-da03-4f48-9281-8462de1e2a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:34.202963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3eda0f-da03-4f48-9281-8462de1e2a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:34.204918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3eda0f-da03-4f48-9281-8462de1e2a8c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:34.221603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 820/1641] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:34.222607 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3eda0f-da03-4f48-9281-8462de1e2a8c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:34.230202 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3f7cb763-e759-48b6-aa68-6f1676d93f1e None None] MFA Rules not processed for user `8da96bed73794fee8f719ec904936164`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:34.231678 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3f7cb763-e759-48b6-aa68-6f1676d93f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:34.234089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3eda0f-da03-4f48-9281-8462de1e2a8c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:34.241841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f245ce0-973d-4e90-8370-849174c5ac82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:34.242858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f245ce0-973d-4e90-8370-849174c5ac82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:34.243210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f245ce0-973d-4e90-8370-849174c5ac82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:34.244179 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f245ce0-973d-4e90-8370-849174c5ac82 admin admin] PATH_INFO: `/v3/projects/d7b18cbe6ebf4707b96c53fc3e7a7c3a/users/25d23dd65cfa4ef4a48f89beb87b51a9/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:34.258819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f245ce0-973d-4e90-8370-849174c5ac82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7b18cbe6ebf4707b96c53fc3e7a7c3a, user_id=25d23dd65cfa4ef4a48f89beb87b51a9, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:34.261662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f245ce0-973d-4e90-8370-849174c5ac82 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:34.274788 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 821/1642] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:34 2022] PUT /identity/v3/projects/d7b18cbe6ebf4707b96c53fc3e7a7c3a/users/25d23dd65cfa4ef4a48f89beb87b51a9/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:34.277905 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f245ce0-973d-4e90-8370-849174c5ac82 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:34.339832 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 822/1643] 10.222.0.64 () {58 vars in 1009 bytes} [Mon May 23 15:41:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:34.341118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d26d21-7251-4f1c-a6b8-8d10288cd883 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:34.341406 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d26d21-7251-4f1c-a6b8-8d10288cd883 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:34.341685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d26d21-7251-4f1c-a6b8-8d10288cd883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:34.385418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a5d26d21-7251-4f1c-a6b8-8d10288cd883 None None] MFA Rules not processed for user `25d23dd65cfa4ef4a48f89beb87b51a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:34.385850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f245ce0-973d-4e90-8370-849174c5ac82 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:34.387176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a5d26d21-7251-4f1c-a6b8-8d10288cd883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:34.392068 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38196233-abeb-4541-96f1-4d5719f76233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:34.395335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38196233-abeb-4541-96f1-4d5719f76233 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:34.395335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38196233-abeb-4541-96f1-4d5719f76233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:34.395335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38196233-abeb-4541-96f1-4d5719f76233 admin admin] PATH_INFO: `/v3/projects/0e18c4b9981342249f01c73ccd2f287f/users/d430d9949e1b446ba54c490ad445855b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:34.396631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38196233-abeb-4541-96f1-4d5719f76233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e18c4b9981342249f01c73ccd2f287f, user_id=d430d9949e1b446ba54c490ad445855b, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:34.399357 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38196233-abeb-4541-96f1-4d5719f76233 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:34.419213 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 822/1644] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:34 2022] PUT /identity/v3/projects/0e18c4b9981342249f01c73ccd2f287f/users/d430d9949e1b446ba54c490ad445855b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:34.420454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38196233-abeb-4541-96f1-4d5719f76233 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:34.429852 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:34.501681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38196233-abeb-4541-96f1-4d5719f76233 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:34.740864 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835911e-f7bf-4090-9805-82c4df1d8dc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:34.740864 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835911e-f7bf-4090-9805-82c4df1d8dc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:34.740864 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835911e-f7bf-4090-9805-82c4df1d8dc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:34.740864 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835911e-f7bf-4090-9805-82c4df1d8dc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:34.742736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 823/1645] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:34 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:34.742736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19d810-636e-4354-85d4-218a6aa7f519 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:34.742736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19d810-636e-4354-85d4-218a6aa7f519 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:34.742736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19d810-636e-4354-85d4-218a6aa7f519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:34.742736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1a19d810-636e-4354-85d4-218a6aa7f519 None None] MFA Rules not processed for user `d430d9949e1b446ba54c490ad445855b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:34.742736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1a19d810-636e-4354-85d4-218a6aa7f519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:34.744048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835911e-f7bf-4090-9805-82c4df1d8dc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:34.744048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835911e-f7bf-4090-9805-82c4df1d8dc2 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:34.744048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 823/1646] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:34.744048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835911e-f7bf-4090-9805-82c4df1d8dc2 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:34.744561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 824/1647] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:34.744561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19d810-636e-4354-85d4-218a6aa7f519 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:34.744561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19d810-636e-4354-85d4-218a6aa7f519 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:34.744561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45321675-e7ad-488d-9db5-f7b465caaf60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:34.747156 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45321675-e7ad-488d-9db5-f7b465caaf60 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:34.747156 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45321675-e7ad-488d-9db5-f7b465caaf60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:34.747156 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45321675-e7ad-488d-9db5-f7b465caaf60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:34.806563 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835911e-f7bf-4090-9805-82c4df1d8dc2 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:34.829294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ba7eb-6c5b-4fe1-b5e3-278ca9aea2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:34.833599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ba7eb-6c5b-4fe1-b5e3-278ca9aea2d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:34.833958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ba7eb-6c5b-4fe1-b5e3-278ca9aea2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:34.834245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ba7eb-6c5b-4fe1-b5e3-278ca9aea2d1 admin admin] PATH_INFO: `/v3/users/48352fabb230456690e795d91115605b` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:34.836430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ba7eb-6c5b-4fe1-b5e3-278ca9aea2d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48352fabb230456690e795d91115605b)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:34.836637 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45321675-e7ad-488d-9db5-f7b465caaf60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:34.837512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: WARNING py.warnings [None req-740ba7eb-6c5b-4fe1-b5e3-278ca9aea2d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:41:34.837512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: warnings.warn(msg) May 23 15:41:34.837512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: May 23 15:41:34.838526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45321675-e7ad-488d-9db5-f7b465caaf60 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:34.841468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ba7eb-6c5b-4fe1-b5e3-278ca9aea2d1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:34.850683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 825/1648] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:34 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:34.853979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86ae3d-250f-4245-bdfe-7f611b46f77c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:34.864997 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86ae3d-250f-4245-bdfe-7f611b46f77c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:34.865351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86ae3d-250f-4245-bdfe-7f611b46f77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:34.909312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8c86ae3d-250f-4245-bdfe-7f611b46f77c None None] MFA Rules not processed for user `916c7e4c45a94167ba39db8c7fec0d19`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:34.910714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8c86ae3d-250f-4245-bdfe-7f611b46f77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:35.005129 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 826/1649] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:35.005129 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86ae3d-250f-4245-bdfe-7f611b46f77c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:35.026321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86ae3d-250f-4245-bdfe-7f611b46f77c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:35.033109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93bfff-8ff4-42ca-a117-32177fb2a749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:35.034155 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93bfff-8ff4-42ca-a117-32177fb2a749 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:35.034268 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93bfff-8ff4-42ca-a117-32177fb2a749 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:35.035111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93bfff-8ff4-42ca-a117-32177fb2a749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:35.097815 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-740ba7eb-6c5b-4fe1-b5e3-278ca9aea2d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48352fabb230456690e795d91115605b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:35.098706 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-740ba7eb-6c5b-4fe1-b5e3-278ca9aea2d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48352fabb230456690e795d91115605b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:35.100235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-740ba7eb-6c5b-4fe1-b5e3-278ca9aea2d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48352fabb230456690e795d91115605b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:35.138263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93bfff-8ff4-42ca-a117-32177fb2a749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:35.138486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-740ba7eb-6c5b-4fe1-b5e3-278ca9aea2d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48352fabb230456690e795d91115605b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:35.144500 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93bfff-8ff4-42ca-a117-32177fb2a749 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:35.161675 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 827/1650] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:35.168365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc7bbf-972f-4bd7-8458-4243262007ab None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:35.168365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc7bbf-972f-4bd7-8458-4243262007ab None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:35.168365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc7bbf-972f-4bd7-8458-4243262007ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:35.190762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:35.192630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c9fc7bbf-972f-4bd7-8458-4243262007ab None None] MFA Rules not processed for user `916c7e4c45a94167ba39db8c7fec0d19`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:35.195289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c9fc7bbf-972f-4bd7-8458-4243262007ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:35.203147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-740ba7eb-6c5b-4fe1-b5e3-278ca9aea2d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48352fabb230456690e795d91115605b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:35.215582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 828/1651] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:35.229270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc7bbf-972f-4bd7-8458-4243262007ab None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:35.231687 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c9fc7bbf-972f-4bd7-8458-4243262007ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:35.262948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc7bbf-972f-4bd7-8458-4243262007ab None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:35.283891 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956ed01-6e45-4f5d-a7ff-ca80595a1c32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:35.284591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956ed01-6e45-4f5d-a7ff-ca80595a1c32 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:35.284909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956ed01-6e45-4f5d-a7ff-ca80595a1c32 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:35.285223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956ed01-6e45-4f5d-a7ff-ca80595a1c32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:35.287314 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5956ed01-6e45-4f5d-a7ff-ca80595a1c32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:35.309000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956ed01-6e45-4f5d-a7ff-ca80595a1c32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:35.310662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956ed01-6e45-4f5d-a7ff-ca80595a1c32 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:35.332337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 829/1652] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:35.345864 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 824/1653] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:41:34 2022] DELETE /identity/v3/users/48352fabb230456690e795d91115605b => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:35.350471 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff226e-5941-4214-b480-8e34583f2a17 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:35.350960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff226e-5941-4214-b480-8e34583f2a17 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:35.351415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff226e-5941-4214-b480-8e34583f2a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:35.383485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3dff226e-5941-4214-b480-8e34583f2a17 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:35.384737 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3dff226e-5941-4214-b480-8e34583f2a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:35.524524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 830/1654] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:35.530909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ba7eb-6c5b-4fe1-b5e3-278ca9aea2d1 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:35.532704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-740ba7eb-6c5b-4fe1-b5e3-278ca9aea2d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:35.567696 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:35.646494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ba7eb-6c5b-4fe1-b5e3-278ca9aea2d1 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:35.650751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec661565-d0f6-43b9-98e1-c1b935726c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:35.651410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec661565-d0f6-43b9-98e1-c1b935726c20 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:35.651844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec661565-d0f6-43b9-98e1-c1b935726c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:35.652537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec661565-d0f6-43b9-98e1-c1b935726c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:35.666577 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec661565-d0f6-43b9-98e1-c1b935726c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:35.668816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec661565-d0f6-43b9-98e1-c1b935726c20 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:35.680540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 825/1655] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:41:36.138566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab79bd6-b13d-4edd-9c13-33b7bca760cb None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:36.139050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab79bd6-b13d-4edd-9c13-33b7bca760cb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:36.140010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab79bd6-b13d-4edd-9c13-33b7bca760cb None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:36.141694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 831/1656] 10.222.0.64 () {54 vars in 912 bytes} [Mon May 23 15:41:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:41:36.157044 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:36.160585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145f5a9-2426-4791-be80-bdf8374f7dc9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:36.160585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145f5a9-2426-4791-be80-bdf8374f7dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:36.160585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145f5a9-2426-4791-be80-bdf8374f7dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:36.205403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-1145f5a9-2426-4791-be80-bdf8374f7dc9 None None] MFA Rules not processed for user `12ceb8a355a44b9fb29918483379cb38`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:36.206727 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab79bd6-b13d-4edd-9c13-33b7bca760cb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:36.207776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1145f5a9-2426-4791-be80-bdf8374f7dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:36.217097 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-eab79bd6-b13d-4edd-9c13-33b7bca760cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:36.242647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 826/1657] 10.222.0.64 () {58 vars in 1009 bytes} [Mon May 23 15:41:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:41:36.250242 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145f5a9-2426-4791-be80-bdf8374f7dc9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:36.271381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145f5a9-2426-4791-be80-bdf8374f7dc9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:36.271635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab79bd6-b13d-4edd-9c13-33b7bca760cb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:36.281030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae0038-ab13-400f-b5f5-8b76d4220bd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:36.284486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f1524-bc10-464b-87d2-35b759f0d183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:36.285474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f1524-bc10-464b-87d2-35b759f0d183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:36.285766 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f1524-bc10-464b-87d2-35b759f0d183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:36.286037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f1524-bc10-464b-87d2-35b759f0d183 admin admin] PATH_INFO: `/v3/users/60cd50c000ed4dbb8eca4a2900bf43a1` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:36.286388 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae0038-ab13-400f-b5f5-8b76d4220bd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:36.287366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae0038-ab13-400f-b5f5-8b76d4220bd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:36.287616 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae0038-ab13-400f-b5f5-8b76d4220bd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:36.292181 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f1524-bc10-464b-87d2-35b759f0d183 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60cd50c000ed4dbb8eca4a2900bf43a1)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:36.341333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f1524-bc10-464b-87d2-35b759f0d183 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:36.351855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae0038-ab13-400f-b5f5-8b76d4220bd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:36.353014 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae0038-ab13-400f-b5f5-8b76d4220bd9 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:36.401944 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 827/1658] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:41:36.493346 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f7bb2-3179-4044-b5e4-6bae5092cb86 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:36.493645 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f7bb2-3179-4044-b5e4-6bae5092cb86 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:36.493931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f7bb2-3179-4044-b5e4-6bae5092cb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:36.543917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e01f7bb2-3179-4044-b5e4-6bae5092cb86 None None] MFA Rules not processed for user `f109fbb1f8d14ffdb6ab4efe06ad9fac`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:36.547342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e01f7bb2-3179-4044-b5e4-6bae5092cb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:36.633777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:36.652264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-df4f1524-bc10-464b-87d2-35b759f0d183 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60cd50c000ed4dbb8eca4a2900bf43a1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:36.680767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-df4f1524-bc10-464b-87d2-35b759f0d183 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60cd50c000ed4dbb8eca4a2900bf43a1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:36.727894 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 828/1659] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:36 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:41:36.739396 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f7bb2-3179-4044-b5e4-6bae5092cb86 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:36.742137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e01f7bb2-3179-4044-b5e4-6bae5092cb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:36.774258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f7bb2-3179-4044-b5e4-6bae5092cb86 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:36.778520 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-df4f1524-bc10-464b-87d2-35b759f0d183 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60cd50c000ed4dbb8eca4a2900bf43a1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:36.779160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-df4f1524-bc10-464b-87d2-35b759f0d183 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60cd50c000ed4dbb8eca4a2900bf43a1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:36.781770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c86b0-67c9-4dbc-8a66-63b2b7ba84aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:36.784224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c86b0-67c9-4dbc-8a66-63b2b7ba84aa admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:36.784224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c86b0-67c9-4dbc-8a66-63b2b7ba84aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:36.784224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c86b0-67c9-4dbc-8a66-63b2b7ba84aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:36.784424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c86b0-67c9-4dbc-8a66-63b2b7ba84aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:36.787544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-df4f1524-bc10-464b-87d2-35b759f0d183 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60cd50c000ed4dbb8eca4a2900bf43a1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:36.787835 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c86b0-67c9-4dbc-8a66-63b2b7ba84aa admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:36.804713 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 832/1660] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:41:36 2022] DELETE /identity/v3/users/60cd50c000ed4dbb8eca4a2900bf43a1 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:36.813198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b10e2-3e56-448b-a8e5-3fe80db8ee2f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:36.813198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b10e2-3e56-448b-a8e5-3fe80db8ee2f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:36.813198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b10e2-3e56-448b-a8e5-3fe80db8ee2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:36.848862 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c14b10e2-3e56-448b-a8e5-3fe80db8ee2f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:36.850181 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c14b10e2-3e56-448b-a8e5-3fe80db8ee2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:36.927881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 829/1661] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:36 2022] POST /identity/v3/projects => generated 403 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:36.932211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c86b0-67c9-4dbc-8a66-63b2b7ba84aa admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:36.939135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1c4c86b0-67c9-4dbc-8a66-63b2b7ba84aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:36.960808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 833/1662] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:36.972427 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b10e2-3e56-448b-a8e5-3fe80db8ee2f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:36.976593 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c14b10e2-3e56-448b-a8e5-3fe80db8ee2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:37.138272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c86b0-67c9-4dbc-8a66-63b2b7ba84aa admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:37.152964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e094be-dfe5-49a8-b482-ccf86ed1d342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:37.153706 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e094be-dfe5-49a8-b482-ccf86ed1d342 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:37.153996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e094be-dfe5-49a8-b482-ccf86ed1d342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:37.154310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e094be-dfe5-49a8-b482-ccf86ed1d342 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:37.157371 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e094be-dfe5-49a8-b482-ccf86ed1d342 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:37.171759 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e094be-dfe5-49a8-b482-ccf86ed1d342 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:37.212013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b10e2-3e56-448b-a8e5-3fe80db8ee2f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:37.351650 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 830/1663] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:36 2022] POST /identity/v3/users => generated 308 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:37.352908 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e094be-dfe5-49a8-b482-ccf86ed1d342 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:37.353396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374b364-ce32-4ef4-907e-b85851d33667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:37.355157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374b364-ce32-4ef4-907e-b85851d33667 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:37.356326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374b364-ce32-4ef4-907e-b85851d33667 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:37.356633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374b364-ce32-4ef4-907e-b85851d33667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:37.356975 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a5e094be-dfe5-49a8-b482-ccf86ed1d342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:37.374302 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e094be-dfe5-49a8-b482-ccf86ed1d342 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:37.383589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92515a92-1937-42be-b14c-598c5b954870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:37.384694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92515a92-1937-42be-b14c-598c5b954870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:37.385184 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92515a92-1937-42be-b14c-598c5b954870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:37.385716 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92515a92-1937-42be-b14c-598c5b954870 admin admin] PATH_INFO: `/v3/projects/f380b8a58cfc4b71b86681885ed58031` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:37.387500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92515a92-1937-42be-b14c-598c5b954870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f380b8a58cfc4b71b86681885ed58031)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:37.390915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92515a92-1937-42be-b14c-598c5b954870 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:37.391057 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374b364-ce32-4ef4-907e-b85851d33667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:37.392381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374b364-ce32-4ef4-907e-b85851d33667 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:37.409930 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 834/1664] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:37.411337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad59e2f-fc42-4f57-8403-ff5c4dacab91 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:37.411692 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad59e2f-fc42-4f57-8403-ff5c4dacab91 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:37.411977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad59e2f-fc42-4f57-8403-ff5c4dacab91 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:37.415023 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 835/1665] 10.222.0.64 () {54 vars in 960 bytes} [Mon May 23 15:41:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:41:37.415682 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad59e2f-fc42-4f57-8403-ff5c4dacab91 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:37.448395 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad59e2f-fc42-4f57-8403-ff5c4dacab91 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:37.458369 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bc581-a331-4c72-8d68-09dab5841215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:37.459257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bc581-a331-4c72-8d68-09dab5841215 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:37.459692 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bc581-a331-4c72-8d68-09dab5841215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:37.460204 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bc581-a331-4c72-8d68-09dab5841215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:37.461096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bc581-a331-4c72-8d68-09dab5841215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:37.463509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bc581-a331-4c72-8d68-09dab5841215 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:37.492802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 836/1666] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:37.493799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4eea0-23ad-4412-85c1-01ff69826cf0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:37.493799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4eea0-23ad-4412-85c1-01ff69826cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:37.493970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4eea0-23ad-4412-85c1-01ff69826cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:37.539241 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-36d4eea0-23ad-4412-85c1-01ff69826cf0 None None] MFA Rules not processed for user `04fbaeb7583a456d81cf79f6b101fb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:37.539927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-36d4eea0-23ad-4412-85c1-01ff69826cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:37.596595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-92515a92-1937-42be-b14c-598c5b954870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f380b8a58cfc4b71b86681885ed58031', 'request_id': 'req-92515a92-1937-42be-b14c-598c5b954870'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:37.606310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-92515a92-1937-42be-b14c-598c5b954870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f380b8a58cfc4b71b86681885ed58031', 'request_id': 'req-92515a92-1937-42be-b14c-598c5b954870'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:37.616734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-92515a92-1937-42be-b14c-598c5b954870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f380b8a58cfc4b71b86681885ed58031', 'request_id': 'req-92515a92-1937-42be-b14c-598c5b954870'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:37.635719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 837/1667] 10.222.0.64 () {58 vars in 1057 bytes} [Mon May 23 15:41:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:37.636707 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4eea0-23ad-4412-85c1-01ff69826cf0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:37.647368 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4eea0-23ad-4412-85c1-01ff69826cf0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:37.680036 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:37.685601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-92515a92-1937-42be-b14c-598c5b954870 admin admin] The token cache is being invalidate because project f380b8a58cfc4b71b86681885ed58031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:41:37.686076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-92515a92-1937-42be-b14c-598c5b954870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:37.687222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-92515a92-1937-42be-b14c-598c5b954870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:37.690082 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 831/1668] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:41:37 2022] DELETE /identity/v3/projects/f380b8a58cfc4b71b86681885ed58031 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:37.693541 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92515a92-1937-42be-b14c-598c5b954870 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:37.695078 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-92515a92-1937-42be-b14c-598c5b954870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:37.735855 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a0a5a-26ec-4538-8466-e045a874e5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:37.736987 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a0a5a-26ec-4538-8466-e045a874e5fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:37.737493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a0a5a-26ec-4538-8466-e045a874e5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:37.737874 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a0a5a-26ec-4538-8466-e045a874e5fd admin admin] PATH_INFO: `/v3/projects/bdff5f53cc9c47648387bf7fc8257dae/users/e081beb51a2142b184cf0587924b31a0/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:37.761614 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a0a5a-26ec-4538-8466-e045a874e5fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdff5f53cc9c47648387bf7fc8257dae, user_id=e081beb51a2142b184cf0587924b31a0, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:37.765590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a0a5a-26ec-4538-8466-e045a874e5fd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:37.812683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92515a92-1937-42be-b14c-598c5b954870 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:37.820094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef0ac0-62f4-4237-9e25-342e8e2a48e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fbaeb7583a456d81cf79f6b101fb1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:37.820926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef0ac0-62f4-4237-9e25-342e8e2a48e2 service placement] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:37.821359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef0ac0-62f4-4237-9e25-342e8e2a48e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:37.821886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef0ac0-62f4-4237-9e25-342e8e2a48e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:37.824066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-18ef0ac0-62f4-4237-9e25-342e8e2a48e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:37.846371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 838/1669] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:37 2022] PUT /identity/v3/projects/bdff5f53cc9c47648387bf7fc8257dae/users/e081beb51a2142b184cf0587924b31a0/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:37.847309 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9753d-c0ec-4ce7-b91f-ca936aee5c20 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:37.847708 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9753d-c0ec-4ce7-b91f-ca936aee5c20 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:37.848096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9753d-c0ec-4ce7-b91f-ca936aee5c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:37.851664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef0ac0-62f4-4237-9e25-342e8e2a48e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:37.854599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef0ac0-62f4-4237-9e25-342e8e2a48e2 service placement] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:37.869952 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-35d9753d-c0ec-4ce7-b91f-ca936aee5c20 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:37.871388 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-35d9753d-c0ec-4ce7-b91f-ca936aee5c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:37.872429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 832/1670] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:37.873928 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef0ac0-62f4-4237-9e25-342e8e2a48e2 service placement] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:37.879616 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-18ef0ac0-62f4-4237-9e25-342e8e2a48e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:37.905659 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef0ac0-62f4-4237-9e25-342e8e2a48e2 service placement] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:38.004579 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:38.032261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 839/1671] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:38.033904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bdc55-6dcc-444e-9f15-b5027b03c31c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:38.036662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bdc55-6dcc-444e-9f15-b5027b03c31c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:38.036662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bdc55-6dcc-444e-9f15-b5027b03c31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:38.062366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7247a5-8962-47d8-8680-cb8ac8e562c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:38.064511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7247a5-8962-47d8-8680-cb8ac8e562c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:38.064966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7247a5-8962-47d8-8680-cb8ac8e562c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:38.065916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7247a5-8962-47d8-8680-cb8ac8e562c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:38.067343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7247a5-8962-47d8-8680-cb8ac8e562c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:38.070777 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7247a5-8962-47d8-8680-cb8ac8e562c7 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:38.090514 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9b6bdc55-6dcc-444e-9f15-b5027b03c31c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:38.103108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9b6bdc55-6dcc-444e-9f15-b5027b03c31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:38.112465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 833/1672] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:38.114300 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7247a5-8962-47d8-8680-cb8ac8e562c7 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:38.127899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7247a5-8962-47d8-8680-cb8ac8e562c7 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:38.130199 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 840/1673] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:38.135618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0814f-36f4-498a-8e28-9508fc543e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:38.136525 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bdc55-6dcc-444e-9f15-b5027b03c31c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:38.138273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0814f-36f4-498a-8e28-9508fc543e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:38.138596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0814f-36f4-498a-8e28-9508fc543e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:38.138881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0814f-36f4-498a-8e28-9508fc543e3f admin admin] PATH_INFO: `/v3/projects/bdff5f53cc9c47648387bf7fc8257dae/users/e081beb51a2142b184cf0587924b31a0/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:38.143709 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0814f-36f4-498a-8e28-9508fc543e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdff5f53cc9c47648387bf7fc8257dae, user_id=e081beb51a2142b184cf0587924b31a0, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:38.153192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0814f-36f4-498a-8e28-9508fc543e3f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:38.155776 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bdc55-6dcc-444e-9f15-b5027b03c31c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:38.162138 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a2b3a-35fc-496b-9b55-f984d0bf3b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:38.163217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a2b3a-35fc-496b-9b55-f984d0bf3b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:38.163746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a2b3a-35fc-496b-9b55-f984d0bf3b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:38.164250 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a2b3a-35fc-496b-9b55-f984d0bf3b37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:38.171332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a2b3a-35fc-496b-9b55-f984d0bf3b37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:41:38.172103 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a2b3a-35fc-496b-9b55-f984d0bf3b37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:38.174083 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a2b3a-35fc-496b-9b55-f984d0bf3b37 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:38.188091 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 834/1674] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:38 2022] PUT /identity/v3/projects/bdff5f53cc9c47648387bf7fc8257dae/users/e081beb51a2142b184cf0587924b31a0/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:38.203681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0814f-36f4-498a-8e28-9508fc543e3f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:38.218776 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 841/1675] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:41:38.392260 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0814f-36f4-498a-8e28-9508fc543e3f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:38.417912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f23d5-318e-4e38-9f17-3a4ee43aadc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:38.418628 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f23d5-318e-4e38-9f17-3a4ee43aadc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:38.418920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f23d5-318e-4e38-9f17-3a4ee43aadc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:38.419212 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f23d5-318e-4e38-9f17-3a4ee43aadc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:38.428799 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f23d5-318e-4e38-9f17-3a4ee43aadc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:38.429150 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f23d5-318e-4e38-9f17-3a4ee43aadc0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:38.556776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 835/1676] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:38 2022] POST /identity/v3/projects => generated 373 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:38.564419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a2b3a-35fc-496b-9b55-f984d0bf3b37 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:38.581825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a2b3a-35fc-496b-9b55-f984d0bf3b37 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:38.672789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f23d5-318e-4e38-9f17-3a4ee43aadc0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:38.675536 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-361f23d5-318e-4e38-9f17-3a4ee43aadc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:38.735573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ada93-cd98-4e74-9517-76c7e582a16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:38.736748 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ada93-cd98-4e74-9517-76c7e582a16d admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:38.737049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ada93-cd98-4e74-9517-76c7e582a16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:38.737373 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ada93-cd98-4e74-9517-76c7e582a16d admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:38.739275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ada93-cd98-4e74-9517-76c7e582a16d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:38.742408 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ada93-cd98-4e74-9517-76c7e582a16d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:38.819041 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f23d5-318e-4e38-9f17-3a4ee43aadc0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:38.828195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:38.829172 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:38.829468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:38.829754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d admin admin] PATH_INFO: `/v3/projects/05c33a8edc9e4b34ad4d7e6da04fa44a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:38.830990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=05c33a8edc9e4b34ad4d7e6da04fa44a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:38.832994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:38.872887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 842/1677] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:38 2022] POST /identity/v3/users => generated 287 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:38.877234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ada93-cd98-4e74-9517-76c7e582a16d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:38.916266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ada93-cd98-4e74-9517-76c7e582a16d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:38.940986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd00d5-e0c1-4f04-a4b9-0de92babc18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:38.941724 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd00d5-e0c1-4f04-a4b9-0de92babc18d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:38.943050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd00d5-e0c1-4f04-a4b9-0de92babc18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:38.943401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd00d5-e0c1-4f04-a4b9-0de92babc18d admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:38.945961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd00d5-e0c1-4f04-a4b9-0de92babc18d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:38.949996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd00d5-e0c1-4f04-a4b9-0de92babc18d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:38.968328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 843/1678] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:38.971994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd00d5-e0c1-4f04-a4b9-0de92babc18d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:38.994025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd00d5-e0c1-4f04-a4b9-0de92babc18d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:39.000251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15884991-83b2-4077-85bb-1aedfa002f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:39.001149 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15884991-83b2-4077-85bb-1aedfa002f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:39.001571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15884991-83b2-4077-85bb-1aedfa002f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:39.001987 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15884991-83b2-4077-85bb-1aedfa002f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:39.002954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15884991-83b2-4077-85bb-1aedfa002f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:39.013018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15884991-83b2-4077-85bb-1aedfa002f75 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:39.017160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 844/1679] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:39.020868 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15884991-83b2-4077-85bb-1aedfa002f75 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:39.047141 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05c33a8edc9e4b34ad4d7e6da04fa44a', 'request_id': 'req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:39.051859 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15884991-83b2-4077-85bb-1aedfa002f75 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:39.075611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05c33a8edc9e4b34ad4d7e6da04fa44a', 'request_id': 'req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:39.098635 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05c33a8edc9e4b34ad4d7e6da04fa44a', 'request_id': 'req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:39.172938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21afa3-823e-4277-a363-66318761525f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:39.174097 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21afa3-823e-4277-a363-66318761525f admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:39.174390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21afa3-823e-4277-a363-66318761525f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:39.174673 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21afa3-823e-4277-a363-66318761525f admin admin] PATH_INFO: `/v3/projects/41af6594a19944c3a7bdcb88856b4a49/users/a60bfca1aff144f385fdd23a44f2fcc8/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:39.210616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21afa3-823e-4277-a363-66318761525f admin admin] RBAC: Authorizing `identity:create_grant(project_id=41af6594a19944c3a7bdcb88856b4a49, user_id=a60bfca1aff144f385fdd23a44f2fcc8, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:39.231483 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d admin admin] The token cache is being invalidate because project 05c33a8edc9e4b34ad4d7e6da04fa44a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:41:39.232499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:39.233853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:39.237311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 836/1680] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:41:38 2022] DELETE /identity/v3/projects/05c33a8edc9e4b34ad4d7e6da04fa44a => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:39.241330 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:39.242979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21afa3-823e-4277-a363-66318761525f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:39.244499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:39.299069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fdec7-37a6-421c-bd02-5cfb0f7dda1d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:39.324415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1a145-1dae-4c35-affa-61c549966ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:39.327685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1a145-1dae-4c35-affa-61c549966ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:39.327979 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1a145-1dae-4c35-affa-61c549966ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:39.328265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1a145-1dae-4c35-affa-61c549966ff6 admin admin] PATH_INFO: `/v3/users/25d23dd65cfa4ef4a48f89beb87b51a9` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:39.332479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1a145-1dae-4c35-affa-61c549966ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d23dd65cfa4ef4a48f89beb87b51a9)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:39.335982 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1a145-1dae-4c35-affa-61c549966ff6 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:39.367806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:39.384652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 845/1681] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:39 2022] PUT /identity/v3/projects/41af6594a19944c3a7bdcb88856b4a49/users/a60bfca1aff144f385fdd23a44f2fcc8/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:39.386460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce2af3-76d7-4503-ae12-34fd3010aa37 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:39.386708 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce2af3-76d7-4503-ae12-34fd3010aa37 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:39.387270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce2af3-76d7-4503-ae12-34fd3010aa37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:39.432567 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:39.442934 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f9ce2af3-76d7-4503-ae12-34fd3010aa37 None None] MFA Rules not processed for user `a60bfca1aff144f385fdd23a44f2fcc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:39.445913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f9ce2af3-76d7-4503-ae12-34fd3010aa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:39.625526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 846/1682] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:41:39.629587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce2af3-76d7-4503-ae12-34fd3010aa37 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:39.631302 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f9ce2af3-76d7-4503-ae12-34fd3010aa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:39.650124 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-59e1a145-1dae-4c35-affa-61c549966ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d23dd65cfa4ef4a48f89beb87b51a9'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:39.651042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-59e1a145-1dae-4c35-affa-61c549966ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d23dd65cfa4ef4a48f89beb87b51a9'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:39.651842 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-59e1a145-1dae-4c35-affa-61c549966ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d23dd65cfa4ef4a48f89beb87b51a9'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:39.666577 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-59e1a145-1dae-4c35-affa-61c549966ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d23dd65cfa4ef4a48f89beb87b51a9'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:39.703901 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-59e1a145-1dae-4c35-affa-61c549966ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d23dd65cfa4ef4a48f89beb87b51a9'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:39.754070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce2af3-76d7-4503-ae12-34fd3010aa37 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:39.775749 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e902e-0ccb-4af5-969c-a414e4a0695f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:39.781401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e902e-0ccb-4af5-969c-a414e4a0695f admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:39.781401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e902e-0ccb-4af5-969c-a414e4a0695f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:39.781401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e902e-0ccb-4af5-969c-a414e4a0695f admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:39.781401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e902e-0ccb-4af5-969c-a414e4a0695f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:39.781401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e902e-0ccb-4af5-969c-a414e4a0695f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:39.834348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 837/1683] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:41:39 2022] DELETE /identity/v3/users/25d23dd65cfa4ef4a48f89beb87b51a9 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:39.898972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 847/1684] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:39 2022] POST /identity/v3/projects => generated 373 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:39.902950 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1a145-1dae-4c35-affa-61c549966ff6 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:39.904330 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-59e1a145-1dae-4c35-affa-61c549966ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:40.047150 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1a145-1dae-4c35-affa-61c549966ff6 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:40.185346 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b413e5a-b6c6-441c-9297-d74aa9d501d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:40.187242 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b413e5a-b6c6-441c-9297-d74aa9d501d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:40.187242 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b413e5a-b6c6-441c-9297-d74aa9d501d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:40.187738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b413e5a-b6c6-441c-9297-d74aa9d501d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:40.188460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b413e5a-b6c6-441c-9297-d74aa9d501d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:40.194174 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b413e5a-b6c6-441c-9297-d74aa9d501d3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:40.318838 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 838/1685] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:39 2022] POST /identity/v3/users => generated 293 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:40.324025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e902e-0ccb-4af5-969c-a414e4a0695f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:40.352355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e902e-0ccb-4af5-969c-a414e4a0695f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:40.370205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9687a-8bc2-45f1-86c8-eee8af153888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:40.371141 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9687a-8bc2-45f1-86c8-eee8af153888 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:40.371598 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9687a-8bc2-45f1-86c8-eee8af153888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:40.372019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9687a-8bc2-45f1-86c8-eee8af153888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:40.373018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9687a-8bc2-45f1-86c8-eee8af153888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:40.375910 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9687a-8bc2-45f1-86c8-eee8af153888 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:40.413279 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 848/1686] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:40.418506 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b413e5a-b6c6-441c-9297-d74aa9d501d3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:40.438770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b413e5a-b6c6-441c-9297-d74aa9d501d3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:40.452890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d29975-e8ac-4640-9235-769569726f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:40.455004 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d29975-e8ac-4640-9235-769569726f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:40.455308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d29975-e8ac-4640-9235-769569726f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:40.455600 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d29975-e8ac-4640-9235-769569726f16 admin admin] PATH_INFO: `/v3/projects/3b012572e6ee4654aabe2ec34b35dcc7/users/df5a635afffb4e4aa3592d15c6a5452e/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:40.483870 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d29975-e8ac-4640-9235-769569726f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b012572e6ee4654aabe2ec34b35dcc7, user_id=df5a635afffb4e4aa3592d15c6a5452e, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:40.489961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d29975-e8ac-4640-9235-769569726f16 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:40.589787 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 839/1687] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:40 2022] PUT /identity/v3/projects/3b012572e6ee4654aabe2ec34b35dcc7/users/df5a635afffb4e4aa3592d15c6a5452e/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:40.594445 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9687a-8bc2-45f1-86c8-eee8af153888 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:40.611116 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9687a-8bc2-45f1-86c8-eee8af153888 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:40.738063 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effe6e1-4f6a-4e05-9a37-f5ff4ab24ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:40.738982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effe6e1-4f6a-4e05-9a37-f5ff4ab24ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:40.739596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effe6e1-4f6a-4e05-9a37-f5ff4ab24ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:40.740175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effe6e1-4f6a-4e05-9a37-f5ff4ab24ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:40.741047 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effe6e1-4f6a-4e05-9a37-f5ff4ab24ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:40.744732 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effe6e1-4f6a-4e05-9a37-f5ff4ab24ac6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:40.756415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 849/1688] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:40.760533 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d29975-e8ac-4640-9235-769569726f16 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:40.779173 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d29975-e8ac-4640-9235-769569726f16 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:40.792253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e2701-28b5-454d-b42c-84d092e9fe42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:40.793726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e2701-28b5-454d-b42c-84d092e9fe42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:40.794228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e2701-28b5-454d-b42c-84d092e9fe42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:40.794645 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e2701-28b5-454d-b42c-84d092e9fe42 admin admin] PATH_INFO: `/v3/projects/3b012572e6ee4654aabe2ec34b35dcc7/users/df5a635afffb4e4aa3592d15c6a5452e/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:40.799942 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e2701-28b5-454d-b42c-84d092e9fe42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b012572e6ee4654aabe2ec34b35dcc7, user_id=df5a635afffb4e4aa3592d15c6a5452e, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:40.803220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e2701-28b5-454d-b42c-84d092e9fe42 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:40.834382 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 840/1689] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:40 2022] PUT /identity/v3/projects/3b012572e6ee4654aabe2ec34b35dcc7/users/df5a635afffb4e4aa3592d15c6a5452e/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:40.844641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a22be-0772-405e-b72c-51cbb8e3edad None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:40.845090 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a22be-0772-405e-b72c-51cbb8e3edad None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:40.845440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a22be-0772-405e-b72c-51cbb8e3edad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:40.853263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:40.908054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-972a22be-0772-405e-b72c-51cbb8e3edad None None] MFA Rules not processed for user `df5a635afffb4e4aa3592d15c6a5452e`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:40.909351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-972a22be-0772-405e-b72c-51cbb8e3edad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:41.019356 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf1e15-a814-4f7a-996e-1fa7d3344ccc None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:41.019817 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf1e15-a814-4f7a-996e-1fa7d3344ccc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:41.020266 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf1e15-a814-4f7a-996e-1fa7d3344ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:41.051836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 850/1690] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:41.052851 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a22be-0772-405e-b72c-51cbb8e3edad None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:41.057614 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-972a22be-0772-405e-b72c-51cbb8e3edad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:41.071283 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-28cf1e15-a814-4f7a-996e-1fa7d3344ccc None None] MFA Rules not processed for user `e081beb51a2142b184cf0587924b31a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:41.074481 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-28cf1e15-a814-4f7a-996e-1fa7d3344ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:41.173209 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:41.184620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a22be-0772-405e-b72c-51cbb8e3edad None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:41.197403 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c0406-519e-47b4-8611-949d02b44fa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:41.198161 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c0406-519e-47b4-8611-949d02b44fa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:41.198449 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c0406-519e-47b4-8611-949d02b44fa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:41.198730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c0406-519e-47b4-8611-949d02b44fa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:41.229194 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c0406-519e-47b4-8611-949d02b44fa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:41.231050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c0406-519e-47b4-8611-949d02b44fa8 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:41.236377 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 841/1691] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:41 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:41:41.257856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 851/1692] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:41.532037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf1e15-a814-4f7a-996e-1fa7d3344ccc None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:41.534267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-28cf1e15-a814-4f7a-996e-1fa7d3344ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:41.562420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:41.623789 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c0406-519e-47b4-8611-949d02b44fa8 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:41.635145 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c0406-519e-47b4-8611-949d02b44fa8 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:41.648481 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b5ebd-02f9-4ad2-8d24-53b1fa6685a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:41.649183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b5ebd-02f9-4ad2-8d24-53b1fa6685a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:41.649631 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b5ebd-02f9-4ad2-8d24-53b1fa6685a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:41.650288 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b5ebd-02f9-4ad2-8d24-53b1fa6685a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:41.652446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-922b5ebd-02f9-4ad2-8d24-53b1fa6685a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:41.655463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf1e15-a814-4f7a-996e-1fa7d3344ccc None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:41.662748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6ecf0-8452-484b-9475-88ac5b3aea3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:41.663812 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6ecf0-8452-484b-9475-88ac5b3aea3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:41.664365 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6ecf0-8452-484b-9475-88ac5b3aea3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:41.669978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6ecf0-8452-484b-9475-88ac5b3aea3a admin admin] PATH_INFO: `/v3/projects/d7b18cbe6ebf4707b96c53fc3e7a7c3a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:41.672569 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6ecf0-8452-484b-9475-88ac5b3aea3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7b18cbe6ebf4707b96c53fc3e7a7c3a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:41.675191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6ecf0-8452-484b-9475-88ac5b3aea3a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:41.795168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b5ebd-02f9-4ad2-8d24-53b1fa6685a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:41.797893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b5ebd-02f9-4ad2-8d24-53b1fa6685a4 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:41.811123 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 852/1693] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:41:41.879077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5da6ecf0-8452-484b-9475-88ac5b3aea3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7b18cbe6ebf4707b96c53fc3e7a7c3a', 'request_id': 'req-5da6ecf0-8452-484b-9475-88ac5b3aea3a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:41.894484 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5da6ecf0-8452-484b-9475-88ac5b3aea3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7b18cbe6ebf4707b96c53fc3e7a7c3a', 'request_id': 'req-5da6ecf0-8452-484b-9475-88ac5b3aea3a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:41.923025 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5da6ecf0-8452-484b-9475-88ac5b3aea3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7b18cbe6ebf4707b96c53fc3e7a7c3a', 'request_id': 'req-5da6ecf0-8452-484b-9475-88ac5b3aea3a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:42.085564 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5da6ecf0-8452-484b-9475-88ac5b3aea3a admin admin] The token cache is being invalidate because project d7b18cbe6ebf4707b96c53fc3e7a7c3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:41:42.085956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5da6ecf0-8452-484b-9475-88ac5b3aea3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:42.087511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5da6ecf0-8452-484b-9475-88ac5b3aea3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:42.090548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 842/1694] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:41:41 2022] DELETE /identity/v3/projects/d7b18cbe6ebf4707b96c53fc3e7a7c3a => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:42.095942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0930f5-814f-4b1c-9388-81b73d4ad267 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:42.096236 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0930f5-814f-4b1c-9388-81b73d4ad267 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:42.096515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0930f5-814f-4b1c-9388-81b73d4ad267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:42.116421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-df0930f5-814f-4b1c-9388-81b73d4ad267 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:42.117886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-df0930f5-814f-4b1c-9388-81b73d4ad267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:42.229114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 853/1695] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:42.243261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17b906-54e1-43e5-bfe8-158e754d25d9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:42.243679 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17b906-54e1-43e5-bfe8-158e754d25d9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:42.244034 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17b906-54e1-43e5-bfe8-158e754d25d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:42.281343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0e17b906-54e1-43e5-bfe8-158e754d25d9 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:42.283543 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0e17b906-54e1-43e5-bfe8-158e754d25d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:42.295077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 843/1696] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:42.299876 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0930f5-814f-4b1c-9388-81b73d4ad267 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:42.316332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0930f5-814f-4b1c-9388-81b73d4ad267 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:42.321928 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd658f7-688f-4008-990b-5825ce8316b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:42.322992 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd658f7-688f-4008-990b-5825ce8316b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:42.331287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd658f7-688f-4008-990b-5825ce8316b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:42.331643 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd658f7-688f-4008-990b-5825ce8316b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:42.333204 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd658f7-688f-4008-990b-5825ce8316b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:41:42.333528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd658f7-688f-4008-990b-5825ce8316b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:42.337748 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd658f7-688f-4008-990b-5825ce8316b2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:42.369560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 854/1697] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:41:42.370795 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17b906-54e1-43e5-bfe8-158e754d25d9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:42.383441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17b906-54e1-43e5-bfe8-158e754d25d9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:42.401712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=88,keystone:DELETE=6,keystone:INSERT=6 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:42.415984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0605d-f890-41a7-8555-c2bc46374dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:42.417047 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0605d-f890-41a7-8555-c2bc46374dca admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:42.417786 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0605d-f890-41a7-8555-c2bc46374dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:42.418656 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0605d-f890-41a7-8555-c2bc46374dca admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:42.419954 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0605d-f890-41a7-8555-c2bc46374dca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:42.422904 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0605d-f890-41a7-8555-c2bc46374dca admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:42.567242 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 844/1698] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:42 2022] POST /identity/v3/projects => generated 385 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:41:42.571442 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd658f7-688f-4008-990b-5825ce8316b2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:42.589604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd658f7-688f-4008-990b-5825ce8316b2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:42.635526 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e97d4f-7ee1-4fad-abb6-00d95e5b5443 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:42.635816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e97d4f-7ee1-4fad-abb6-00d95e5b5443 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:42.636247 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e97d4f-7ee1-4fad-abb6-00d95e5b5443 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:42.637798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 845/1699] 10.222.0.64 () {54 vars in 921 bytes} [Mon May 23 15:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:41:42.670652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbf50c-ac88-4198-97e0-aaaf6ce40f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:42.671348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbf50c-ac88-4198-97e0-aaaf6ce40f24 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:42.671847 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbf50c-ac88-4198-97e0-aaaf6ce40f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:42.672333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbf50c-ac88-4198-97e0-aaaf6ce40f24 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:42.673294 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbf50c-ac88-4198-97e0-aaaf6ce40f24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:42.675585 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbf50c-ac88-4198-97e0-aaaf6ce40f24 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:42.692834 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c6e82-a6d1-4942-95de-2ee6727fd00f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:42.693331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c6e82-a6d1-4942-95de-2ee6727fd00f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:42.694862 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c6e82-a6d1-4942-95de-2ee6727fd00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:42.741949 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a78c6e82-a6d1-4942-95de-2ee6727fd00f None None] MFA Rules not processed for user `04fbaeb7583a456d81cf79f6b101fb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:42.747481 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:42.754652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a78c6e82-a6d1-4942-95de-2ee6727fd00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:42.808278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 855/1700] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:42 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:42.813350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbf50c-ac88-4198-97e0-aaaf6ce40f24 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:42.834890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbf50c-ac88-4198-97e0-aaaf6ce40f24 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:42.854015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19b70c-0368-4d99-8b1f-d4ebd8431181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:42.855030 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19b70c-0368-4d99-8b1f-d4ebd8431181 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:42.856974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19b70c-0368-4d99-8b1f-d4ebd8431181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:42.857409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19b70c-0368-4d99-8b1f-d4ebd8431181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:42.858487 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19b70c-0368-4d99-8b1f-d4ebd8431181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:42.865000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19b70c-0368-4d99-8b1f-d4ebd8431181 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:42.895298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 856/1701] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:42.898723 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19b70c-0368-4d99-8b1f-d4ebd8431181 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:42.927714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19b70c-0368-4d99-8b1f-d4ebd8431181 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:42.940232 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 846/1702] 10.222.0.64 () {58 vars in 1018 bytes} [Mon May 23 15:41:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:41:42.949073 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e27d9-641f-4672-88a6-6f01086230f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:42.950886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e27d9-641f-4672-88a6-6f01086230f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:42.951362 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e27d9-641f-4672-88a6-6f01086230f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:42.953172 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e27d9-641f-4672-88a6-6f01086230f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:42.954309 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e27d9-641f-4672-88a6-6f01086230f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:42.958312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e27d9-641f-4672-88a6-6f01086230f2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:42.979646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c6e82-a6d1-4942-95de-2ee6727fd00f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:42.986566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 857/1703] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:42.988933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a78c6e82-a6d1-4942-95de-2ee6727fd00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:42.989168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e27d9-641f-4672-88a6-6f01086230f2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:43.009609 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c6e82-a6d1-4942-95de-2ee6727fd00f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:43.016360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e27d9-641f-4672-88a6-6f01086230f2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:43.022561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17bc49-1148-4fc1-b652-963b3340e486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:43.024970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17bc49-1148-4fc1-b652-963b3340e486 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:43.025421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17bc49-1148-4fc1-b652-963b3340e486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:43.025922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17bc49-1148-4fc1-b652-963b3340e486 admin admin] PATH_INFO: `/v3/projects/9f964dc442d44b118374d6e44e13795c/users/3180f42b648243b7bc8aa0a26f024763/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:43.036533 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda2adb-840b-4d9a-95f6-0d165dcc7a91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fbaeb7583a456d81cf79f6b101fb1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:43.037986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda2adb-840b-4d9a-95f6-0d165dcc7a91 service placement] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:43.037986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda2adb-840b-4d9a-95f6-0d165dcc7a91 service placement] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:43.037986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda2adb-840b-4d9a-95f6-0d165dcc7a91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:43.048308 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17bc49-1148-4fc1-b652-963b3340e486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f964dc442d44b118374d6e44e13795c, user_id=3180f42b648243b7bc8aa0a26f024763, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:43.052265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17bc49-1148-4fc1-b652-963b3340e486 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:43.081909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda2adb-840b-4d9a-95f6-0d165dcc7a91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:43.083721 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda2adb-840b-4d9a-95f6-0d165dcc7a91 service placement] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:43.113159 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 847/1704] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:43.166548 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 858/1705] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:42 2022] PUT /identity/v3/projects/9f964dc442d44b118374d6e44e13795c/users/3180f42b648243b7bc8aa0a26f024763/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:43.178014 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a5c92-1d4d-48aa-b805-e18a016c3f58 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:43.179004 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a5c92-1d4d-48aa-b805-e18a016c3f58 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:43.179361 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a5c92-1d4d-48aa-b805-e18a016c3f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:43.251942 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-912a5c92-1d4d-48aa-b805-e18a016c3f58 None None] MFA Rules not processed for user `3180f42b648243b7bc8aa0a26f024763`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:43.253844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-912a5c92-1d4d-48aa-b805-e18a016c3f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:43.386668 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 848/1706] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:43.387614 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17bc49-1148-4fc1-b652-963b3340e486 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:43.503552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a5c92-1d4d-48aa-b805-e18a016c3f58 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:43.505437 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-912a5c92-1d4d-48aa-b805-e18a016c3f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:43.550006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17bc49-1148-4fc1-b652-963b3340e486 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:43.559635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d1a23-adf3-4c8a-a688-f504d11d222d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:43.560420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d1a23-adf3-4c8a-a688-f504d11d222d admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:43.561890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d1a23-adf3-4c8a-a688-f504d11d222d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:43.562180 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d1a23-adf3-4c8a-a688-f504d11d222d admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:43.562890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d1a23-adf3-4c8a-a688-f504d11d222d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:43.565033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d1a23-adf3-4c8a-a688-f504d11d222d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:43.620048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a5c92-1d4d-48aa-b805-e18a016c3f58 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:43.630707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f77bc-4acd-4557-932e-74e4fc5356d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:43.631535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f77bc-4acd-4557-932e-74e4fc5356d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:43.632085 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f77bc-4acd-4557-932e-74e4fc5356d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:43.633367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f77bc-4acd-4557-932e-74e4fc5356d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:43.640465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7d6f77bc-4acd-4557-932e-74e4fc5356d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:43.672245 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 859/1707] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:43 2022] POST /identity/v3/projects => generated 383 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:43.679082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d1a23-adf3-4c8a-a688-f504d11d222d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:43.779084 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:43.829461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d1a23-adf3-4c8a-a688-f504d11d222d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:43.836345 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9083b31-2fd6-4a24-90a5-2c1891b1729e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:43.839112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9083b31-2fd6-4a24-90a5-2c1891b1729e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:43.839112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9083b31-2fd6-4a24-90a5-2c1891b1729e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:43.839112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9083b31-2fd6-4a24-90a5-2c1891b1729e admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:43.839112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9083b31-2fd6-4a24-90a5-2c1891b1729e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:43.847112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9083b31-2fd6-4a24-90a5-2c1891b1729e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:43.867333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f77bc-4acd-4557-932e-74e4fc5356d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:43.869047 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f77bc-4acd-4557-932e-74e4fc5356d6 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:43.910516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 849/1708] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:41:44.001475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 860/1709] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:43 2022] POST /identity/v3/users => generated 298 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:41:44.002986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f77bc-4acd-4557-932e-74e4fc5356d6 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:44.023763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:44.030461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f77bc-4acd-4557-932e-74e4fc5356d6 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:44.058317 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd4b24-6d46-4693-9dfe-a2645f6c6e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:44.060201 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd4b24-6d46-4693-9dfe-a2645f6c6e7b admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:44.069592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd4b24-6d46-4693-9dfe-a2645f6c6e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:44.069592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd4b24-6d46-4693-9dfe-a2645f6c6e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:44.070025 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd4b24-6d46-4693-9dfe-a2645f6c6e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:44.071775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd4b24-6d46-4693-9dfe-a2645f6c6e7b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:44.094236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 850/1710] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:44.095821 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9083b31-2fd6-4a24-90a5-2c1891b1729e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:44.116577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9083b31-2fd6-4a24-90a5-2c1891b1729e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:44.143622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a11a2-87dd-4e0c-9b5f-369d6e079401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:44.144723 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a11a2-87dd-4e0c-9b5f-369d6e079401 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:44.146097 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a11a2-87dd-4e0c-9b5f-369d6e079401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:44.146489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a11a2-87dd-4e0c-9b5f-369d6e079401 admin admin] PATH_INFO: `/v3/projects/108e7633965f465da1c2a0536f330c44/users/90a4bd8d252b4c50b43bc9d79cd3ece2/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:44.193208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a11a2-87dd-4e0c-9b5f-369d6e079401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=108e7633965f465da1c2a0536f330c44, user_id=90a4bd8d252b4c50b43bc9d79cd3ece2, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:44.200723 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a11a2-87dd-4e0c-9b5f-369d6e079401 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:44.317042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 861/1711] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:44 2022] PUT /identity/v3/projects/108e7633965f465da1c2a0536f330c44/users/90a4bd8d252b4c50b43bc9d79cd3ece2/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:44.321076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd4b24-6d46-4693-9dfe-a2645f6c6e7b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:44.332543 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd4b24-6d46-4693-9dfe-a2645f6c6e7b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:44.491853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532ee1e-c7e2-4551-bcbd-dde826546096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:44.492874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532ee1e-c7e2-4551-bcbd-dde826546096 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:44.493887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532ee1e-c7e2-4551-bcbd-dde826546096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:44.494308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532ee1e-c7e2-4551-bcbd-dde826546096 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:44.495277 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532ee1e-c7e2-4551-bcbd-dde826546096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:44.497494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532ee1e-c7e2-4551-bcbd-dde826546096 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:44.520890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 851/1712] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:44.525282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a11a2-87dd-4e0c-9b5f-369d6e079401 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:44.561755 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a11a2-87dd-4e0c-9b5f-369d6e079401 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:44.567624 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c48900-ce82-4049-886c-4d1bc5ce10d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:44.568837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c48900-ce82-4049-886c-4d1bc5ce10d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:44.569247 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c48900-ce82-4049-886c-4d1bc5ce10d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:44.569528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c48900-ce82-4049-886c-4d1bc5ce10d4 admin admin] PATH_INFO: `/v3/projects/108e7633965f465da1c2a0536f330c44/users/90a4bd8d252b4c50b43bc9d79cd3ece2/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:44.572848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c48900-ce82-4049-886c-4d1bc5ce10d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=108e7633965f465da1c2a0536f330c44, user_id=90a4bd8d252b4c50b43bc9d79cd3ece2, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:44.575664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c48900-ce82-4049-886c-4d1bc5ce10d4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:44.598373 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 862/1713] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:44 2022] PUT /identity/v3/projects/108e7633965f465da1c2a0536f330c44/users/90a4bd8d252b4c50b43bc9d79cd3ece2/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:44.601197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef0b5f-a455-4a1c-9a9f-ed9fb032479b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:44.601197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef0b5f-a455-4a1c-9a9f-ed9fb032479b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:44.601376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef0b5f-a455-4a1c-9a9f-ed9fb032479b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:44.637529 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf1c75-de5a-4fca-bdfd-0e2fd4fcebcb None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:44.637823 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf1c75-de5a-4fca-bdfd-0e2fd4fcebcb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:44.638108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf1c75-de5a-4fca-bdfd-0e2fd4fcebcb None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:44.641909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 863/1714] 10.222.0.64 () {54 vars in 921 bytes} [Mon May 23 15:41:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:41:44.660626 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b3ef0b5f-a455-4a1c-9a9f-ed9fb032479b None None] MFA Rules not processed for user `90a4bd8d252b4c50b43bc9d79cd3ece2`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:44.662775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b3ef0b5f-a455-4a1c-9a9f-ed9fb032479b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:44.823101 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 852/1715] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:44.937386 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf1c75-de5a-4fca-bdfd-0e2fd4fcebcb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:44.939873 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8edf1c75-de5a-4fca-bdfd-0e2fd4fcebcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:45.113446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf1c75-de5a-4fca-bdfd-0e2fd4fcebcb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:45.122281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ae1df-e424-4b4b-ab11-2222cae9529b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:45.124989 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ae1df-e424-4b4b-ab11-2222cae9529b service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:45.124989 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ae1df-e424-4b4b-ab11-2222cae9529b service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:45.124989 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ae1df-e424-4b4b-ab11-2222cae9529b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:45.162530 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd511e53-dd4f-48bc-b6f2-bd49f9981084 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:45.163711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd511e53-dd4f-48bc-b6f2-bd49f9981084 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:45.164229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd511e53-dd4f-48bc-b6f2-bd49f9981084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:45.222572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fd511e53-dd4f-48bc-b6f2-bd49f9981084 None None] MFA Rules not processed for user `04fbaeb7583a456d81cf79f6b101fb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:45.228768 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fd511e53-dd4f-48bc-b6f2-bd49f9981084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:45.337557 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ae1df-e424-4b4b-ab11-2222cae9529b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:45.344003 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ae1df-e424-4b4b-ab11-2222cae9529b service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:45.375513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 864/1716] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:45.401627 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ae1df-e424-4b4b-ab11-2222cae9529b service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:45.402288 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d18ae1df-e424-4b4b-ab11-2222cae9529b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:45.448212 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 853/1717] 10.222.0.64 () {58 vars in 1018 bytes} [Mon May 23 15:41:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:45.459861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105885e-a405-4056-9e2e-66be9f2f1948 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:45.459861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105885e-a405-4056-9e2e-66be9f2f1948 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:45.459861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105885e-a405-4056-9e2e-66be9f2f1948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:45.514412 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:45.524002 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-1105885e-a405-4056-9e2e-66be9f2f1948 None None] MFA Rules not processed for user `a2f1d435732b4ff0a60599309c66eeb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:45.525900 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1105885e-a405-4056-9e2e-66be9f2f1948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:45.546794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ae1df-e424-4b4b-ab11-2222cae9529b service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:45.552886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f009b8-9f98-4c8e-b2b5-d37db1c8d052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:45.554992 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f009b8-9f98-4c8e-b2b5-d37db1c8d052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:45.555686 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f009b8-9f98-4c8e-b2b5-d37db1c8d052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:45.556113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f009b8-9f98-4c8e-b2b5-d37db1c8d052 admin admin] PATH_INFO: `/v3/users/f109fbb1f8d14ffdb6ab4efe06ad9fac` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:45.563804 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f009b8-9f98-4c8e-b2b5-d37db1c8d052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f109fbb1f8d14ffdb6ab4efe06ad9fac)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:45.566031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f009b8-9f98-4c8e-b2b5-d37db1c8d052 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:45.641977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 854/1718] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:45.641977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105885e-a405-4056-9e2e-66be9f2f1948 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:45.645525 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1105885e-a405-4056-9e2e-66be9f2f1948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:45.663358 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:45.668616 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105885e-a405-4056-9e2e-66be9f2f1948 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:45.676092 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b594f3-aeb0-43f2-a312-2af7ac0baae8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fbaeb7583a456d81cf79f6b101fb1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:45.677303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b594f3-aeb0-43f2-a312-2af7ac0baae8 service placement] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:45.677594 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b594f3-aeb0-43f2-a312-2af7ac0baae8 service placement] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:45.681214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b594f3-aeb0-43f2-a312-2af7ac0baae8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:45.697146 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b594f3-aeb0-43f2-a312-2af7ac0baae8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:45.697146 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b594f3-aeb0-43f2-a312-2af7ac0baae8 service placement] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:45.707688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 855/1719] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:45.709397 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199a524-7c39-4da2-a52f-a54c36c47943 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:45.709772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199a524-7c39-4da2-a52f-a54c36c47943 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:45.710131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199a524-7c39-4da2-a52f-a54c36c47943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:45.750286 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-41f009b8-9f98-4c8e-b2b5-d37db1c8d052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f109fbb1f8d14ffdb6ab4efe06ad9fac'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:45.753695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a199a524-7c39-4da2-a52f-a54c36c47943 None None] MFA Rules not processed for user `5236a93fc3a84d27bf3d150e85eb2589`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:45.753695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a199a524-7c39-4da2-a52f-a54c36c47943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:45.790506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-41f009b8-9f98-4c8e-b2b5-d37db1c8d052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f109fbb1f8d14ffdb6ab4efe06ad9fac'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:45.859566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-41f009b8-9f98-4c8e-b2b5-d37db1c8d052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f109fbb1f8d14ffdb6ab4efe06ad9fac'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:45.860259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-41f009b8-9f98-4c8e-b2b5-d37db1c8d052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f109fbb1f8d14ffdb6ab4efe06ad9fac'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:45.870403 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-41f009b8-9f98-4c8e-b2b5-d37db1c8d052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f109fbb1f8d14ffdb6ab4efe06ad9fac'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:45.882148 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 856/1720] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:45.883625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237b56d-d030-42c7-96b0-3af85c85d012 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:45.883997 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237b56d-d030-42c7-96b0-3af85c85d012 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:45.884672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237b56d-d030-42c7-96b0-3af85c85d012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:45.896996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 865/1721] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:41:45 2022] DELETE /identity/v3/users/f109fbb1f8d14ffdb6ab4efe06ad9fac => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:45.910794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f009b8-9f98-4c8e-b2b5-d37db1c8d052 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:45.914449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b237b56d-d030-42c7-96b0-3af85c85d012 None None] MFA Rules not processed for user `a2f1d435732b4ff0a60599309c66eeb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:45.914584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-41f009b8-9f98-4c8e-b2b5-d37db1c8d052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:45.926317 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b237b56d-d030-42c7-96b0-3af85c85d012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:46.140515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 857/1722] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:46.147520 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f009b8-9f98-4c8e-b2b5-d37db1c8d052 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:46.148070 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237b56d-d030-42c7-96b0-3af85c85d012 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:46.149146 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b237b56d-d030-42c7-96b0-3af85c85d012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:46.155523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d2a39-6b39-4fd6-89e3-b0358e86cfa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:46.156222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d2a39-6b39-4fd6-89e3-b0358e86cfa4 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:46.156496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d2a39-6b39-4fd6-89e3-b0358e86cfa4 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:46.156781 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d2a39-6b39-4fd6-89e3-b0358e86cfa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:46.178094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237b56d-d030-42c7-96b0-3af85c85d012 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:46.187966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf5178-6b37-4575-aeab-601f4211de84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:46.188640 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf5178-6b37-4575-aeab-601f4211de84 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:46.188927 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf5178-6b37-4575-aeab-601f4211de84 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:46.191348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf5178-6b37-4575-aeab-601f4211de84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:46.197971 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d9cf5178-6b37-4575-aeab-601f4211de84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:46.223341 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf5178-6b37-4575-aeab-601f4211de84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:46.226312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf5178-6b37-4575-aeab-601f4211de84 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:46.238979 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 858/1723] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:46.258465 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d2a39-6b39-4fd6-89e3-b0358e86cfa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:46.260198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d2a39-6b39-4fd6-89e3-b0358e86cfa4 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:46.271427 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 866/1724] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:46.351453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf5178-6b37-4575-aeab-601f4211de84 service nova] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:46.353103 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d9cf5178-6b37-4575-aeab-601f4211de84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:46.526042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf5178-6b37-4575-aeab-601f4211de84 service nova] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:46.544798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2f66c-0227-44c6-bd54-b643280cb318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:46.545849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2f66c-0227-44c6-bd54-b643280cb318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:46.546274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2f66c-0227-44c6-bd54-b643280cb318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:46.546498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2f66c-0227-44c6-bd54-b643280cb318 admin admin] PATH_INFO: `/v3/projects/bcd7eb803f544d04bb7a1eca4b7ad913` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:46.548888 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2f66c-0227-44c6-bd54-b643280cb318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcd7eb803f544d04bb7a1eca4b7ad913)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:46.556761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2f66c-0227-44c6-bd54-b643280cb318 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:46.579501 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:46.796128 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0b2f66c-0227-44c6-bd54-b643280cb318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcd7eb803f544d04bb7a1eca4b7ad913', 'request_id': 'req-c0b2f66c-0227-44c6-bd54-b643280cb318'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:46.810627 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0b2f66c-0227-44c6-bd54-b643280cb318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcd7eb803f544d04bb7a1eca4b7ad913', 'request_id': 'req-c0b2f66c-0227-44c6-bd54-b643280cb318'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:46.822901 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0b2f66c-0227-44c6-bd54-b643280cb318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcd7eb803f544d04bb7a1eca4b7ad913', 'request_id': 'req-c0b2f66c-0227-44c6-bd54-b643280cb318'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:46.940293 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0b2f66c-0227-44c6-bd54-b643280cb318 admin admin] The token cache is being invalidate because project bcd7eb803f544d04bb7a1eca4b7ad913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:41:46.940604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0b2f66c-0227-44c6-bd54-b643280cb318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:46.941324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0b2f66c-0227-44c6-bd54-b643280cb318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:46.943279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 859/1725] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:41:46 2022] DELETE /identity/v3/projects/bcd7eb803f544d04bb7a1eca4b7ad913 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:46.946958 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d2a39-6b39-4fd6-89e3-b0358e86cfa4 service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:46.960005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d2a39-6b39-4fd6-89e3-b0358e86cfa4 service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:47.007441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:47.099005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f35b56-8653-4822-9e27-78cfb1cf71ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:47.099005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f35b56-8653-4822-9e27-78cfb1cf71ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:47.099005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f35b56-8653-4822-9e27-78cfb1cf71ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:47.099005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f35b56-8653-4822-9e27-78cfb1cf71ec admin admin] PATH_INFO: `/v3/users/e081beb51a2142b184cf0587924b31a0` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:47.099611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f35b56-8653-4822-9e27-78cfb1cf71ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=e081beb51a2142b184cf0587924b31a0)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:47.099891 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f35b56-8653-4822-9e27-78cfb1cf71ec admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:47.423812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-95f35b56-8653-4822-9e27-78cfb1cf71ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e081beb51a2142b184cf0587924b31a0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:47.457088 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-95f35b56-8653-4822-9e27-78cfb1cf71ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e081beb51a2142b184cf0587924b31a0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:47.479382 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee638c3-2ba7-49f9-a00a-c6862e838910 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:47.479686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee638c3-2ba7-49f9-a00a-c6862e838910 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:47.479983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee638c3-2ba7-49f9-a00a-c6862e838910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:47.514991 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0ee638c3-2ba7-49f9-a00a-c6862e838910 None None] MFA Rules not processed for user `a2f1d435732b4ff0a60599309c66eeb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:47.517897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0ee638c3-2ba7-49f9-a00a-c6862e838910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:47.606916 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-95f35b56-8653-4822-9e27-78cfb1cf71ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e081beb51a2142b184cf0587924b31a0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:47.607730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-95f35b56-8653-4822-9e27-78cfb1cf71ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e081beb51a2142b184cf0587924b31a0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:47.635547 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-95f35b56-8653-4822-9e27-78cfb1cf71ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e081beb51a2142b184cf0587924b31a0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:47.642021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 860/1726] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:47.650800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 867/1727] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:41:46 2022] DELETE /identity/v3/users/e081beb51a2142b184cf0587924b31a0 => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:47.893003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:48.079104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee638c3-2ba7-49f9-a00a-c6862e838910 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:48.084519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0ee638c3-2ba7-49f9-a00a-c6862e838910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:48.109108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee638c3-2ba7-49f9-a00a-c6862e838910 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:48.263661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f87322-aa14-4b9a-b6da-76b1589b55c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:48.267180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f87322-aa14-4b9a-b6da-76b1589b55c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:48.267990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f87322-aa14-4b9a-b6da-76b1589b55c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:48.269325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f87322-aa14-4b9a-b6da-76b1589b55c2 admin admin] PATH_INFO: `/v3/projects/bdff5f53cc9c47648387bf7fc8257dae` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:48.273047 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f87322-aa14-4b9a-b6da-76b1589b55c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdff5f53cc9c47648387bf7fc8257dae)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:48.285653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f87322-aa14-4b9a-b6da-76b1589b55c2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:48.460960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a7f87322-aa14-4b9a-b6da-76b1589b55c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdff5f53cc9c47648387bf7fc8257dae', 'request_id': 'req-a7f87322-aa14-4b9a-b6da-76b1589b55c2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:48.477238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a7f87322-aa14-4b9a-b6da-76b1589b55c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdff5f53cc9c47648387bf7fc8257dae', 'request_id': 'req-a7f87322-aa14-4b9a-b6da-76b1589b55c2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:48.489182 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a7f87322-aa14-4b9a-b6da-76b1589b55c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdff5f53cc9c47648387bf7fc8257dae', 'request_id': 'req-a7f87322-aa14-4b9a-b6da-76b1589b55c2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:48.595413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a7f87322-aa14-4b9a-b6da-76b1589b55c2 admin admin] The token cache is being invalidate because project bdff5f53cc9c47648387bf7fc8257dae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:41:48.595874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a7f87322-aa14-4b9a-b6da-76b1589b55c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:48.597351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a7f87322-aa14-4b9a-b6da-76b1589b55c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:48.603505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 861/1728] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:41:48 2022] DELETE /identity/v3/projects/bdff5f53cc9c47648387bf7fc8257dae => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:48.609134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3283271-b7e5-41d2-9fe5-7984b14835e6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:48.610038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3283271-b7e5-41d2-9fe5-7984b14835e6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:48.610544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3283271-b7e5-41d2-9fe5-7984b14835e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:48.650846 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d3283271-b7e5-41d2-9fe5-7984b14835e6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:48.652017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d3283271-b7e5-41d2-9fe5-7984b14835e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:48.712091 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:48.790899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 868/1729] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:48.800325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6c5a8-2b41-4409-829e-6e7f47ae9c91 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:48.800890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6c5a8-2b41-4409-829e-6e7f47ae9c91 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:48.801518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6c5a8-2b41-4409-829e-6e7f47ae9c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:48.848871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-84c6c5a8-2b41-4409-829e-6e7f47ae9c91 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:48.850420 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-84c6c5a8-2b41-4409-829e-6e7f47ae9c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:48.875662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 862/1730] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:48.877999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3283271-b7e5-41d2-9fe5-7984b14835e6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:48.896796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3283271-b7e5-41d2-9fe5-7984b14835e6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:48.912145 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572cb20-c6fd-418c-b5f3-b84ebbd80305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:48.912846 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572cb20-c6fd-418c-b5f3-b84ebbd80305 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:48.915415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572cb20-c6fd-418c-b5f3-b84ebbd80305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:48.916033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572cb20-c6fd-418c-b5f3-b84ebbd80305 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:48.918354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572cb20-c6fd-418c-b5f3-b84ebbd80305 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:41:48.919210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572cb20-c6fd-418c-b5f3-b84ebbd80305 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:48.928909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572cb20-c6fd-418c-b5f3-b84ebbd80305 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:48.959816 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 869/1731] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:41:48.961313 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6c5a8-2b41-4409-829e-6e7f47ae9c91 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:48.973173 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6c5a8-2b41-4409-829e-6e7f47ae9c91 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:48.979784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092260b-fb5e-4bf7-972c-e1f8f1157e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:48.980880 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092260b-fb5e-4bf7-972c-e1f8f1157e8c admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:48.981361 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092260b-fb5e-4bf7-972c-e1f8f1157e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:48.982083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092260b-fb5e-4bf7-972c-e1f8f1157e8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:48.983735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092260b-fb5e-4bf7-972c-e1f8f1157e8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:48.987027 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092260b-fb5e-4bf7-972c-e1f8f1157e8c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:49.107686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 863/1732] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:48 2022] POST /identity/v3/projects => generated 367 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:49.108939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572cb20-c6fd-418c-b5f3-b84ebbd80305 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:49.115932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:49.124163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572cb20-c6fd-418c-b5f3-b84ebbd80305 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:49.277203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968ccb2-6669-4ffb-8ed8-ece6a3226e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:49.278141 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968ccb2-6669-4ffb-8ed8-ece6a3226e1c admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:49.278610 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968ccb2-6669-4ffb-8ed8-ece6a3226e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:49.279042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968ccb2-6669-4ffb-8ed8-ece6a3226e1c admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:49.280585 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968ccb2-6669-4ffb-8ed8-ece6a3226e1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:49.306977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968ccb2-6669-4ffb-8ed8-ece6a3226e1c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:49.446226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 870/1733] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:49 2022] POST /identity/v3/users => generated 284 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:49.447023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092260b-fb5e-4bf7-972c-e1f8f1157e8c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:49.468632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092260b-fb5e-4bf7-972c-e1f8f1157e8c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:49.517938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac41d3-d6c5-4d42-bcb3-0216b5ada512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:49.518844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac41d3-d6c5-4d42-bcb3-0216b5ada512 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:49.519631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac41d3-d6c5-4d42-bcb3-0216b5ada512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:49.520096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac41d3-d6c5-4d42-bcb3-0216b5ada512 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:49.521975 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac41d3-d6c5-4d42-bcb3-0216b5ada512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:49.524544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac41d3-d6c5-4d42-bcb3-0216b5ada512 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:49.553000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 864/1734] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:41:49.564422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968ccb2-6669-4ffb-8ed8-ece6a3226e1c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:49.577573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968ccb2-6669-4ffb-8ed8-ece6a3226e1c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:49.593700 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a42a68-e7c4-4bca-96b5-72f4dd121b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:49.594619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a42a68-e7c4-4bca-96b5-72f4dd121b98 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:49.595055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a42a68-e7c4-4bca-96b5-72f4dd121b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:49.595500 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a42a68-e7c4-4bca-96b5-72f4dd121b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:49.600171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a42a68-e7c4-4bca-96b5-72f4dd121b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:49.602262 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a42a68-e7c4-4bca-96b5-72f4dd121b98 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:49.633386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac41d3-d6c5-4d42-bcb3-0216b5ada512 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:49.634562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 871/1735] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:49.650488 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac41d3-d6c5-4d42-bcb3-0216b5ada512 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:49.668381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77dcc5f-bc5d-4e95-8125-5ce90f36de9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:49.669682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dcc5f-bc5d-4e95-8125-5ce90f36de9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:49.670249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dcc5f-bc5d-4e95-8125-5ce90f36de9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:49.670686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dcc5f-bc5d-4e95-8125-5ce90f36de9b admin admin] PATH_INFO: `/v3/projects/cd2adc567771483baf43fc619e21049f/users/e5ed3264c1a449a0b531a53044926d51/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:49.736686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77dcc5f-bc5d-4e95-8125-5ce90f36de9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd2adc567771483baf43fc619e21049f, user_id=e5ed3264c1a449a0b531a53044926d51, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:49.739783 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77dcc5f-bc5d-4e95-8125-5ce90f36de9b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:49.859657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 865/1736] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:49 2022] PUT /identity/v3/projects/cd2adc567771483baf43fc619e21049f/users/e5ed3264c1a449a0b531a53044926d51/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:49.869384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360b434-dfd7-4106-9ba3-17f60288e475 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:49.869679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360b434-dfd7-4106-9ba3-17f60288e475 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:49.869963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360b434-dfd7-4106-9ba3-17f60288e475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:49.898331 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f360b434-dfd7-4106-9ba3-17f60288e475 None None] MFA Rules not processed for user `e5ed3264c1a449a0b531a53044926d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:49.899787 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f360b434-dfd7-4106-9ba3-17f60288e475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:49.984139 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 872/1737] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:49.988723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77dcc5f-bc5d-4e95-8125-5ce90f36de9b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:50.054790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77dcc5f-bc5d-4e95-8125-5ce90f36de9b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:50.059530 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b82d84-b8db-4f56-bdef-1dabe803fd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:50.060203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b82d84-b8db-4f56-bdef-1dabe803fd83 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:50.060495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b82d84-b8db-4f56-bdef-1dabe803fd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:50.060892 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b82d84-b8db-4f56-bdef-1dabe803fd83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:50.061650 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b82d84-b8db-4f56-bdef-1dabe803fd83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:50.070918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b82d84-b8db-4f56-bdef-1dabe803fd83 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:50.192763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 866/1738] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:49 2022] POST /identity/v3/projects => generated 367 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:50.197672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360b434-dfd7-4106-9ba3-17f60288e475 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:50.210515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360b434-dfd7-4106-9ba3-17f60288e475 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:50.281688 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbdd24-8ce0-4d2c-a1d8-aa0e3d1b8674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:50.282781 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbdd24-8ce0-4d2c-a1d8-aa0e3d1b8674 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:50.283248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbdd24-8ce0-4d2c-a1d8-aa0e3d1b8674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:50.283681 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbdd24-8ce0-4d2c-a1d8-aa0e3d1b8674 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:50.284788 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbdd24-8ce0-4d2c-a1d8-aa0e3d1b8674 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:50.286892 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbdd24-8ce0-4d2c-a1d8-aa0e3d1b8674 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:50.375276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 873/1739] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:50 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:50.379849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b82d84-b8db-4f56-bdef-1dabe803fd83 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:50.393943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b82d84-b8db-4f56-bdef-1dabe803fd83 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:50.400646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e88e97-b329-40b2-a8b1-6939e4314f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:50.401411 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e88e97-b329-40b2-a8b1-6939e4314f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:50.401706 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e88e97-b329-40b2-a8b1-6939e4314f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:50.401989 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e88e97-b329-40b2-a8b1-6939e4314f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:50.403112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e88e97-b329-40b2-a8b1-6939e4314f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:50.405247 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e88e97-b329-40b2-a8b1-6939e4314f58 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:50.418893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 867/1740] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:50.423808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbdd24-8ce0-4d2c-a1d8-aa0e3d1b8674 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:50.442858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726b24a-ec85-4ca6-a696-9cfab518c2e1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:50.449529 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726b24a-ec85-4ca6-a696-9cfab518c2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:50.450074 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726b24a-ec85-4ca6-a696-9cfab518c2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:50.462868 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbdd24-8ce0-4d2c-a1d8-aa0e3d1b8674 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:50.481597 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0e744-695a-495d-843b-d684b2e6ca51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:50.481597 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0e744-695a-495d-843b-d684b2e6ca51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:50.481597 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0e744-695a-495d-843b-d684b2e6ca51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:50.481597 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0e744-695a-495d-843b-d684b2e6ca51 admin admin] PATH_INFO: `/v3/projects/9d630080d58a44b6b9dca2eef708b5f3/users/031528b314c749dcb9961cfaad302909/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:50.485355 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7726b24a-ec85-4ca6-a696-9cfab518c2e1 None None] MFA Rules not processed for user `5236a93fc3a84d27bf3d150e85eb2589`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:50.486847 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7726b24a-ec85-4ca6-a696-9cfab518c2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:50.501713 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0e744-695a-495d-843b-d684b2e6ca51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d630080d58a44b6b9dca2eef708b5f3, user_id=031528b314c749dcb9961cfaad302909, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:50.504712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0e744-695a-495d-843b-d684b2e6ca51 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:50.562091 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 868/1741] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:50.591401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 874/1742] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:50 2022] PUT /identity/v3/projects/9d630080d58a44b6b9dca2eef708b5f3/users/031528b314c749dcb9961cfaad302909/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:50.600240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726b24a-ec85-4ca6-a696-9cfab518c2e1 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:50.612662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726b24a-ec85-4ca6-a696-9cfab518c2e1 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:50.679564 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d331a8c-a78c-4cb7-9afa-170787bcc09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:50.680256 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d331a8c-a78c-4cb7-9afa-170787bcc09c admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:50.680696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d331a8c-a78c-4cb7-9afa-170787bcc09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:50.681151 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d331a8c-a78c-4cb7-9afa-170787bcc09c admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:50.682028 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d331a8c-a78c-4cb7-9afa-170787bcc09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:50.684597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d331a8c-a78c-4cb7-9afa-170787bcc09c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:50.696322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 869/1743] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:50.700551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0e744-695a-495d-843b-d684b2e6ca51 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:50.715785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0e744-695a-495d-843b-d684b2e6ca51 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:50.729512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d3ff1-c320-4e7e-af8e-b5bcd484047c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:50.730871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d3ff1-c320-4e7e-af8e-b5bcd484047c admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:50.731359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d3ff1-c320-4e7e-af8e-b5bcd484047c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:50.731804 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d3ff1-c320-4e7e-af8e-b5bcd484047c admin admin] PATH_INFO: `/v3/projects/9d630080d58a44b6b9dca2eef708b5f3/users/031528b314c749dcb9961cfaad302909/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:50.736307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d3ff1-c320-4e7e-af8e-b5bcd484047c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d630080d58a44b6b9dca2eef708b5f3, user_id=031528b314c749dcb9961cfaad302909, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:50.744424 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d3ff1-c320-4e7e-af8e-b5bcd484047c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:50.762924 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 875/1744] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:50 2022] PUT /identity/v3/projects/9d630080d58a44b6b9dca2eef708b5f3/users/031528b314c749dcb9961cfaad302909/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:50.775841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b4cbb-5c3c-48ef-b6fa-81d9c3fe3d23 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:50.776211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b4cbb-5c3c-48ef-b6fa-81d9c3fe3d23 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:50.776528 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b4cbb-5c3c-48ef-b6fa-81d9c3fe3d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:50.812084 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-844b4cbb-5c3c-48ef-b6fa-81d9c3fe3d23 None None] MFA Rules not processed for user `031528b314c749dcb9961cfaad302909`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:50.813648 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-844b4cbb-5c3c-48ef-b6fa-81d9c3fe3d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:50.842691 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:50.928254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 870/1745] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:50 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:50.939890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d3ff1-c320-4e7e-af8e-b5bcd484047c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:50.941807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2b2d3ff1-c320-4e7e-af8e-b5bcd484047c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:50.992303 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:51.031420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d3ff1-c320-4e7e-af8e-b5bcd484047c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:51.038386 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39104aea-5b25-47c3-b570-56632c7e47bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:51.039069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39104aea-5b25-47c3-b570-56632c7e47bc service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:51.039362 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39104aea-5b25-47c3-b570-56632c7e47bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:51.044134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39104aea-5b25-47c3-b570-56632c7e47bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:51.051215 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39104aea-5b25-47c3-b570-56632c7e47bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:51.052663 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39104aea-5b25-47c3-b570-56632c7e47bc service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:51.064105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 876/1746] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:50 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:51.199315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b4cbb-5c3c-48ef-b6fa-81d9c3fe3d23 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:51.200869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-844b4cbb-5c3c-48ef-b6fa-81d9c3fe3d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:51.232418 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b4cbb-5c3c-48ef-b6fa-81d9c3fe3d23 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:51.232679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e8e32-8a42-4f16-95a9-a76f7cc9b102 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:51.233789 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e8e32-8a42-4f16-95a9-a76f7cc9b102 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:51.234277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e8e32-8a42-4f16-95a9-a76f7cc9b102 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:51.236061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 877/1747] 10.222.0.64 () {54 vars in 912 bytes} [Mon May 23 15:41:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:41:51.242136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16117257-c3c2-493a-9dae-1579d4c66894 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:51.242799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16117257-c3c2-493a-9dae-1579d4c66894 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:51.243027 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60347cf9-5013-49a1-80f2-641ed1f3b6c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:51.243534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16117257-c3c2-493a-9dae-1579d4c66894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:51.243714 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60347cf9-5013-49a1-80f2-641ed1f3b6c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:51.244028 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60347cf9-5013-49a1-80f2-641ed1f3b6c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:51.244306 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60347cf9-5013-49a1-80f2-641ed1f3b6c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:51.266438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-16117257-c3c2-493a-9dae-1579d4c66894 None None] MFA Rules not processed for user `12ceb8a355a44b9fb29918483379cb38`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:51.268810 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-16117257-c3c2-493a-9dae-1579d4c66894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:51.322095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60347cf9-5013-49a1-80f2-641ed1f3b6c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:51.325117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60347cf9-5013-49a1-80f2-641ed1f3b6c6 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:51.338100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 871/1748] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:51.343513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 878/1749] 10.222.0.64 () {58 vars in 1009 bytes} [Mon May 23 15:41:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:41:51.362017 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60347cf9-5013-49a1-80f2-641ed1f3b6c6 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:51.362900 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-60347cf9-5013-49a1-80f2-641ed1f3b6c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:51.375737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60347cf9-5013-49a1-80f2-641ed1f3b6c6 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:51.380222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef41c0-71c8-4e7c-b011-cfb7767f35d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:51.381109 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef41c0-71c8-4e7c-b011-cfb7767f35d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:51.385521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef41c0-71c8-4e7c-b011-cfb7767f35d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:51.385521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef41c0-71c8-4e7c-b011-cfb7767f35d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:51.391015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef41c0-71c8-4e7c-b011-cfb7767f35d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:51.392477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef41c0-71c8-4e7c-b011-cfb7767f35d6 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:51.401636 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 872/1750] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:52.167100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:52.733435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29171947-d316-48f9-81fa-0f715d9aaa41 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:52.734258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29171947-d316-48f9-81fa-0f715d9aaa41 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:52.735801 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29171947-d316-48f9-81fa-0f715d9aaa41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:52.753425 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-29171947-d316-48f9-81fa-0f715d9aaa41 None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:52.754638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-29171947-d316-48f9-81fa-0f715d9aaa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:52.865377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 879/1751] 10.222.0.64 () {58 vars in 1009 bytes} [Mon May 23 15:41:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:53.225843 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef41c0-71c8-4e7c-b011-cfb7767f35d6 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:53.227488 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2eef41c0-71c8-4e7c-b011-cfb7767f35d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:53.332416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef41c0-71c8-4e7c-b011-cfb7767f35d6 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:53.338477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e1271-9672-49e8-9b45-4f5015f3c21c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:53.339271 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e1271-9672-49e8-9b45-4f5015f3c21c service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:53.339639 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e1271-9672-49e8-9b45-4f5015f3c21c service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:53.340039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e1271-9672-49e8-9b45-4f5015f3c21c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:53.362362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e1271-9672-49e8-9b45-4f5015f3c21c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:53.364000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e1271-9672-49e8-9b45-4f5015f3c21c service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:53.383509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 873/1752] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:41:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 15:41:55.848670 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29171947-d316-48f9-81fa-0f715d9aaa41 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:55.862227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-29171947-d316-48f9-81fa-0f715d9aaa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:55.898116 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:55.970882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29171947-d316-48f9-81fa-0f715d9aaa41 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:55.976750 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c90958-4a80-4797-936a-18f0d4cded02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:55.977788 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c90958-4a80-4797-936a-18f0d4cded02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:55.978141 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c90958-4a80-4797-936a-18f0d4cded02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:55.978499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c90958-4a80-4797-936a-18f0d4cded02 admin admin] PATH_INFO: `/v3/users/49f723f9ef414cd79a6167dcd852ffe1` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:55.979903 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c90958-4a80-4797-936a-18f0d4cded02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49f723f9ef414cd79a6167dcd852ffe1)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:55.982101 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c90958-4a80-4797-936a-18f0d4cded02 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:56.149913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-02c90958-4a80-4797-936a-18f0d4cded02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49f723f9ef414cd79a6167dcd852ffe1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:56.163876 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-02c90958-4a80-4797-936a-18f0d4cded02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49f723f9ef414cd79a6167dcd852ffe1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:56.264154 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-02c90958-4a80-4797-936a-18f0d4cded02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49f723f9ef414cd79a6167dcd852ffe1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:56.264932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-02c90958-4a80-4797-936a-18f0d4cded02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49f723f9ef414cd79a6167dcd852ffe1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:56.273002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-02c90958-4a80-4797-936a-18f0d4cded02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49f723f9ef414cd79a6167dcd852ffe1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:56.301335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 880/1753] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:41:55 2022] DELETE /identity/v3/users/49f723f9ef414cd79a6167dcd852ffe1 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:56.303447 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e011c6e-15cc-43cb-86b6-61a7dbf4a71a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:56.303966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e011c6e-15cc-43cb-86b6-61a7dbf4a71a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:56.304197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e011c6e-15cc-43cb-86b6-61a7dbf4a71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:56.332627 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6e011c6e-15cc-43cb-86b6-61a7dbf4a71a None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:56.333958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6e011c6e-15cc-43cb-86b6-61a7dbf4a71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:56.400351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:56.464916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 874/1754] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:56.489128 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c90958-4a80-4797-936a-18f0d4cded02 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:56.490561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-02c90958-4a80-4797-936a-18f0d4cded02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:56.581328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c90958-4a80-4797-936a-18f0d4cded02 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:56.586024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30706102-e45f-4552-8c6d-2ae5c302700d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:56.587267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30706102-e45f-4552-8c6d-2ae5c302700d service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:56.587564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30706102-e45f-4552-8c6d-2ae5c302700d service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:56.587843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30706102-e45f-4552-8c6d-2ae5c302700d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:56.598933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30706102-e45f-4552-8c6d-2ae5c302700d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:56.600366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30706102-e45f-4552-8c6d-2ae5c302700d service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:56.617258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 881/1755] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:41:57.352131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e011c6e-15cc-43cb-86b6-61a7dbf4a71a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:57.353941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6e011c6e-15cc-43cb-86b6-61a7dbf4a71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:57.368763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e011c6e-15cc-43cb-86b6-61a7dbf4a71a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:57.374904 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1f765-aa15-4668-8a88-f854c65e195d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:57.375789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1f765-aa15-4668-8a88-f854c65e195d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:57.376077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1f765-aa15-4668-8a88-f854c65e195d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:57.381543 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1f765-aa15-4668-8a88-f854c65e195d admin admin] PATH_INFO: `/v3/projects/e3a7e6bce7fc48f996703cb5eb3533fb` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:57.383337 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1f765-aa15-4668-8a88-f854c65e195d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a7e6bce7fc48f996703cb5eb3533fb)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:57.385744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1f765-aa15-4668-8a88-f854c65e195d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:57.565315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f8f1f765-aa15-4668-8a88-f854c65e195d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a7e6bce7fc48f996703cb5eb3533fb', 'request_id': 'req-f8f1f765-aa15-4668-8a88-f854c65e195d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:57.586144 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f8f1f765-aa15-4668-8a88-f854c65e195d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a7e6bce7fc48f996703cb5eb3533fb', 'request_id': 'req-f8f1f765-aa15-4668-8a88-f854c65e195d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:57.595589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f8f1f765-aa15-4668-8a88-f854c65e195d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a7e6bce7fc48f996703cb5eb3533fb', 'request_id': 'req-f8f1f765-aa15-4668-8a88-f854c65e195d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:57.700247 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f8f1f765-aa15-4668-8a88-f854c65e195d admin admin] The token cache is being invalidate because project e3a7e6bce7fc48f996703cb5eb3533fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:41:57.700543 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f8f1f765-aa15-4668-8a88-f854c65e195d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:57.701961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f8f1f765-aa15-4668-8a88-f854c65e195d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:57.704996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 875/1756] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:41:57 2022] DELETE /identity/v3/projects/e3a7e6bce7fc48f996703cb5eb3533fb => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:57.716264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5448b2e-1eec-4ef0-a8e2-71d3700e687d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:57.716791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5448b2e-1eec-4ef0-a8e2-71d3700e687d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:57.717364 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5448b2e-1eec-4ef0-a8e2-71d3700e687d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:57.749840 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d5448b2e-1eec-4ef0-a8e2-71d3700e687d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:57.751606 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d5448b2e-1eec-4ef0-a8e2-71d3700e687d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:57.820293 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:57.880780 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fec33-bfbc-4e76-b59b-7cec5f0c5da8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:57.880979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 882/1757] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:57.881672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fec33-bfbc-4e76-b59b-7cec5f0c5da8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:57.881963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fec33-bfbc-4e76-b59b-7cec5f0c5da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:57.906468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-954fec33-bfbc-4e76-b59b-7cec5f0c5da8 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:57.907738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-954fec33-bfbc-4e76-b59b-7cec5f0c5da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:57.922788 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 876/1758] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:57.927234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5448b2e-1eec-4ef0-a8e2-71d3700e687d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:57.946100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5448b2e-1eec-4ef0-a8e2-71d3700e687d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:57.953373 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ce36f-6d1a-48c8-9854-2674cc3c37ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:57.954213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ce36f-6d1a-48c8-9854-2674cc3c37ad admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:57.955244 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ce36f-6d1a-48c8-9854-2674cc3c37ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:57.955244 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ce36f-6d1a-48c8-9854-2674cc3c37ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:57.955964 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ce36f-6d1a-48c8-9854-2674cc3c37ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:41:57.956280 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ce36f-6d1a-48c8-9854-2674cc3c37ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:57.965127 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ce36f-6d1a-48c8-9854-2674cc3c37ad admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:57.980229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 883/1759] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:41:57.984841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fec33-bfbc-4e76-b59b-7cec5f0c5da8 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:57.994861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fec33-bfbc-4e76-b59b-7cec5f0c5da8 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:58.001394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada166cd-2203-439b-9054-1bfe06022cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:58.002162 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada166cd-2203-439b-9054-1bfe06022cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:58.002574 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada166cd-2203-439b-9054-1bfe06022cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:58.002934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada166cd-2203-439b-9054-1bfe06022cc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:58.003655 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada166cd-2203-439b-9054-1bfe06022cc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:58.005927 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada166cd-2203-439b-9054-1bfe06022cc0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:58.157903 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 877/1760] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:57 2022] POST /identity/v3/projects => generated 367 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:58.162215 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ce36f-6d1a-48c8-9854-2674cc3c37ad admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:58.173082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ce36f-6d1a-48c8-9854-2674cc3c37ad admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:58.277789 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8bdff-9b74-49ed-ad2b-322f92f20c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:58.278804 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8bdff-9b74-49ed-ad2b-322f92f20c4d admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:58.279773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8bdff-9b74-49ed-ad2b-322f92f20c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:58.280138 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8bdff-9b74-49ed-ad2b-322f92f20c4d admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:58.281079 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8bdff-9b74-49ed-ad2b-322f92f20c4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:58.283534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8bdff-9b74-49ed-ad2b-322f92f20c4d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:58.386912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 884/1761] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:58 2022] POST /identity/v3/users => generated 284 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:41:58.390929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada166cd-2203-439b-9054-1bfe06022cc0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:58.401200 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada166cd-2203-439b-9054-1bfe06022cc0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:58.407090 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8bdff-9b74-49ed-ad2b-322f92f20c4d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:58.408621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4794a-1fc9-417f-974a-a8cd5067272a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:58.409488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-efe8bdff-9b74-49ed-ad2b-322f92f20c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:58.410029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4794a-1fc9-417f-974a-a8cd5067272a admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:58.410434 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4794a-1fc9-417f-974a-a8cd5067272a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:58.410781 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4794a-1fc9-417f-974a-a8cd5067272a admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:58.411567 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b4794a-1fc9-417f-974a-a8cd5067272a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:58.413219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b4794a-1fc9-417f-974a-a8cd5067272a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:58.425506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8bdff-9b74-49ed-ad2b-322f92f20c4d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:58.428748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 878/1762] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:58.432507 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4794a-1fc9-417f-974a-a8cd5067272a admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:58.433963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072ed64-bd79-40ae-9cd7-306d674fa16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:58.435048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072ed64-bd79-40ae-9cd7-306d674fa16d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:58.435494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072ed64-bd79-40ae-9cd7-306d674fa16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:58.435978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072ed64-bd79-40ae-9cd7-306d674fa16d admin admin] PATH_INFO: `/v3/users/e5ed3264c1a449a0b531a53044926d51` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:58.437606 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072ed64-bd79-40ae-9cd7-306d674fa16d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5ed3264c1a449a0b531a53044926d51)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:58.440739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072ed64-bd79-40ae-9cd7-306d674fa16d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:58.447869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4794a-1fc9-417f-974a-a8cd5067272a admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:58.457876 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08b6fb-fd34-498f-aabb-3368454aab26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:58.458566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08b6fb-fd34-498f-aabb-3368454aab26 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:58.458872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08b6fb-fd34-498f-aabb-3368454aab26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:58.459155 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08b6fb-fd34-498f-aabb-3368454aab26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:58.459896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08b6fb-fd34-498f-aabb-3368454aab26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:58.461623 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08b6fb-fd34-498f-aabb-3368454aab26 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:58.474974 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 879/1763] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:58.479966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08b6fb-fd34-498f-aabb-3368454aab26 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:58.491525 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08b6fb-fd34-498f-aabb-3368454aab26 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:58.496725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86de184-2462-419d-8c73-c9b3097d98dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:58.497669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86de184-2462-419d-8c73-c9b3097d98dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:58.497976 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86de184-2462-419d-8c73-c9b3097d98dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:58.498445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86de184-2462-419d-8c73-c9b3097d98dd admin admin] PATH_INFO: `/v3/projects/a854dc79e12c435cbec1dc539e1414e9/users/5dd2e26041364f1fbf7e738d552b8415/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:58.515797 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86de184-2462-419d-8c73-c9b3097d98dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a854dc79e12c435cbec1dc539e1414e9, user_id=5dd2e26041364f1fbf7e738d552b8415, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:58.519243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86de184-2462-419d-8c73-c9b3097d98dd admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:58.679230 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 880/1764] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:58 2022] PUT /identity/v3/projects/a854dc79e12c435cbec1dc539e1414e9/users/5dd2e26041364f1fbf7e738d552b8415/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:41:58.692001 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2e7c9-7dca-4b39-ae8e-28a00425be51 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:58.692304 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2e7c9-7dca-4b39-ae8e-28a00425be51 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:58.692590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2e7c9-7dca-4b39-ae8e-28a00425be51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:58.759766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e7b2e7c9-7dca-4b39-ae8e-28a00425be51 None None] MFA Rules not processed for user `5dd2e26041364f1fbf7e738d552b8415`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:58.763392 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e7b2e7c9-7dca-4b39-ae8e-28a00425be51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:58.823108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:58.872794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 881/1765] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:41:58.875730 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2e7c9-7dca-4b39-ae8e-28a00425be51 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:59.022207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2e7c9-7dca-4b39-ae8e-28a00425be51 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:59.027031 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e5af4-b7e7-47cb-8cd8-e6093709a5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:59.027726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e5af4-b7e7-47cb-8cd8-e6093709a5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:59.028021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e5af4-b7e7-47cb-8cd8-e6093709a5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:59.028308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e5af4-b7e7-47cb-8cd8-e6093709a5f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:59.033485 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e5af4-b7e7-47cb-8cd8-e6093709a5f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:59.042260 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e5af4-b7e7-47cb-8cd8-e6093709a5f2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:59.180365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7072ed64-bd79-40ae-9cd7-306d674fa16d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5ed3264c1a449a0b531a53044926d51'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:59.194019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7072ed64-bd79-40ae-9cd7-306d674fa16d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5ed3264c1a449a0b531a53044926d51'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:59.406587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 882/1766] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:41:58 2022] POST /identity/v3/projects => generated 373 bytes in 529 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:59.408514 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e5af4-b7e7-47cb-8cd8-e6093709a5f2 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:59.441460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7072ed64-bd79-40ae-9cd7-306d674fa16d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5ed3264c1a449a0b531a53044926d51'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:59.442446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7072ed64-bd79-40ae-9cd7-306d674fa16d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5ed3264c1a449a0b531a53044926d51'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:59.451233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7072ed64-bd79-40ae-9cd7-306d674fa16d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5ed3264c1a449a0b531a53044926d51'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:41:59.459037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 885/1767] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:41:58 2022] DELETE /identity/v3/users/e5ed3264c1a449a0b531a53044926d51 => generated 0 bytes in 1051 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:41:59.462214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8d2e2-60e3-4ec7-88e7-78179c8b3006 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:59.462505 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8d2e2-60e3-4ec7-88e7-78179c8b3006 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:59.462784 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8d2e2-60e3-4ec7-88e7-78179c8b3006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:59.488344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ebc8d2e2-60e3-4ec7-88e7-78179c8b3006 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:59.488565 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e5af4-b7e7-47cb-8cd8-e6093709a5f2 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:59.490174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ebc8d2e2-60e3-4ec7-88e7-78179c8b3006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:59.528604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:59.561919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab809f-5d6d-4bd9-9144-8969923fb228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:59.562595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab809f-5d6d-4bd9-9144-8969923fb228 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:59.565099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab809f-5d6d-4bd9-9144-8969923fb228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:59.565254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 886/1768] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:41:59.565394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab809f-5d6d-4bd9-9144-8969923fb228 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:59.565394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab809f-5d6d-4bd9-9144-8969923fb228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:59.565869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab809f-5d6d-4bd9-9144-8969923fb228 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:59.574121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8d2e2-60e3-4ec7-88e7-78179c8b3006 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:59.575862 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ebc8d2e2-60e3-4ec7-88e7-78179c8b3006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:59.643936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8d2e2-60e3-4ec7-88e7-78179c8b3006 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:59.647998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0abd0-92e9-44e0-8e2c-400461306cc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:59.648957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0abd0-92e9-44e0-8e2c-400461306cc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:59.649087 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0abd0-92e9-44e0-8e2c-400461306cc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:59.649374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0abd0-92e9-44e0-8e2c-400461306cc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:59.664143 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0abd0-92e9-44e0-8e2c-400461306cc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:59.665687 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0abd0-92e9-44e0-8e2c-400461306cc5 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:59.679035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 887/1769] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:41:59.725244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 883/1770] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:41:59 2022] POST /identity/v3/users => generated 293 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:41:59.728322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0abd0-92e9-44e0-8e2c-400461306cc5 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:59.729946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a3e0abd0-92e9-44e0-8e2c-400461306cc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:59.746550 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0abd0-92e9-44e0-8e2c-400461306cc5 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:59.751638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a72c6-0971-4690-9cfb-5313be3e01ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:59.752416 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a72c6-0971-4690-9cfb-5313be3e01ea admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:59.752779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a72c6-0971-4690-9cfb-5313be3e01ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:59.753165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a72c6-0971-4690-9cfb-5313be3e01ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:59.753987 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a72c6-0971-4690-9cfb-5313be3e01ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:59.755725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a72c6-0971-4690-9cfb-5313be3e01ea admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:59.770749 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 888/1771] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:59.772081 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab809f-5d6d-4bd9-9144-8969923fb228 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:59.793630 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab809f-5d6d-4bd9-9144-8969923fb228 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:59.802699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234b295-cc51-4b41-b61c-dc8be0dc20ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:59.803930 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234b295-cc51-4b41-b61c-dc8be0dc20ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:59.804317 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234b295-cc51-4b41-b61c-dc8be0dc20ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:59.804702 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234b295-cc51-4b41-b61c-dc8be0dc20ce admin admin] PATH_INFO: `/v3/projects/409b84a992cd492c8a2b3d17b15887c9/users/9a8e6f332408432f9670f98c7cb39c67/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:59.861203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234b295-cc51-4b41-b61c-dc8be0dc20ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=409b84a992cd492c8a2b3d17b15887c9, user_id=9a8e6f332408432f9670f98c7cb39c67, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:59.872912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234b295-cc51-4b41-b61c-dc8be0dc20ce admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:59.902918 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a72c6-0971-4690-9cfb-5313be3e01ea admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:59.905209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-084a72c6-0971-4690-9cfb-5313be3e01ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:00.009108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a72c6-0971-4690-9cfb-5313be3e01ea admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:00.009108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1dbd72-c8fb-4fe3-a3c5-05d91e79e601 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:00.009108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1dbd72-c8fb-4fe3-a3c5-05d91e79e601 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:00.009108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1dbd72-c8fb-4fe3-a3c5-05d91e79e601 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:00.009623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1dbd72-c8fb-4fe3-a3c5-05d91e79e601 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:00.012982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ed1dbd72-c8fb-4fe3-a3c5-05d91e79e601 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:00.093664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 884/1772] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:41:59 2022] PUT /identity/v3/projects/409b84a992cd492c8a2b3d17b15887c9/users/9a8e6f332408432f9670f98c7cb39c67/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:00.094723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234b295-cc51-4b41-b61c-dc8be0dc20ce admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:00.130191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1dbd72-c8fb-4fe3-a3c5-05d91e79e601 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:00.131667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1dbd72-c8fb-4fe3-a3c5-05d91e79e601 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:00.158419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 889/1773] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:41:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:00.314273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234b295-cc51-4b41-b61c-dc8be0dc20ce admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:00.342747 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4c371-76bb-4529-84a4-98b544275994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:00.343513 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4c371-76bb-4529-84a4-98b544275994 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:00.343806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4c371-76bb-4529-84a4-98b544275994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:00.344088 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4c371-76bb-4529-84a4-98b544275994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:00.346585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4c371-76bb-4529-84a4-98b544275994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:00.351406 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4c371-76bb-4529-84a4-98b544275994 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:00.387033 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 885/1774] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:00.392000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1dbd72-c8fb-4fe3-a3c5-05d91e79e601 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:00.422776 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1dbd72-c8fb-4fe3-a3c5-05d91e79e601 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:00.440157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6b181-df43-4bef-9e40-8e1847394d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:00.443388 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6b181-df43-4bef-9e40-8e1847394d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:00.443813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6b181-df43-4bef-9e40-8e1847394d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:00.444206 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6b181-df43-4bef-9e40-8e1847394d95 admin admin] PATH_INFO: `/v3/projects/409b84a992cd492c8a2b3d17b15887c9/users/9a8e6f332408432f9670f98c7cb39c67/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:00.448802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6b181-df43-4bef-9e40-8e1847394d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=409b84a992cd492c8a2b3d17b15887c9, user_id=9a8e6f332408432f9670f98c7cb39c67, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:00.454035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6b181-df43-4bef-9e40-8e1847394d95 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:00.473441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 890/1775] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:00 2022] PUT /identity/v3/projects/409b84a992cd492c8a2b3d17b15887c9/users/9a8e6f332408432f9670f98c7cb39c67/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:00.487193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a0666-cabb-4f9f-b543-7fb0ffff6bc5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:00.487693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a0666-cabb-4f9f-b543-7fb0ffff6bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:00.487779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a0666-cabb-4f9f-b543-7fb0ffff6bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:00.530487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d79a0666-cabb-4f9f-b543-7fb0ffff6bc5 None None] MFA Rules not processed for user `9a8e6f332408432f9670f98c7cb39c67`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:00.537470 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d79a0666-cabb-4f9f-b543-7fb0ffff6bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:00.691592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 886/1776] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:00.894847 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6b181-df43-4bef-9e40-8e1847394d95 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:00.896829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a7d6b181-df43-4bef-9e40-8e1847394d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:01.036337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6b181-df43-4bef-9e40-8e1847394d95 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:01.042774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:01.044155 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:01.045401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:01.045912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] PATH_INFO: `/v3/projects/cd2adc567771483baf43fc619e21049f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:01.047584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd2adc567771483baf43fc619e21049f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:01.048138 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c5baf-5611-4d3c-ba82-e898b4e7647f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:01.048666 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c5baf-5611-4d3c-ba82-e898b4e7647f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:01.049252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c5baf-5611-4d3c-ba82-e898b4e7647f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:01.049531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:42:01.049531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: warnings.warn(msg) May 23 15:42:01.049531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: May 23 15:42:01.051074 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:01.122681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-793c5baf-5611-4d3c-ba82-e898b4e7647f None None] MFA Rules not processed for user `12ceb8a355a44b9fb29918483379cb38`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:01.130288 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-793c5baf-5611-4d3c-ba82-e898b4e7647f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:01.141519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:01.290078 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd2adc567771483baf43fc619e21049f', 'request_id': 'req-1542ffdf-96b1-4ff0-ac72-6b3b12068306'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:01.324429 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd2adc567771483baf43fc619e21049f', 'request_id': 'req-1542ffdf-96b1-4ff0-ac72-6b3b12068306'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:01.355921 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 887/1777] 10.222.0.69 () {58 vars in 1015 bytes} [Mon May 23 15:42:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:01.359181 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c5baf-5611-4d3c-ba82-e898b4e7647f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:01.362956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-793c5baf-5611-4d3c-ba82-e898b4e7647f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:01.493493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c5baf-5611-4d3c-ba82-e898b4e7647f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:01.504400 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd2adc567771483baf43fc619e21049f', 'request_id': 'req-1542ffdf-96b1-4ff0-ac72-6b3b12068306'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:01.532698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] The token cache is being invalidate because project cd2adc567771483baf43fc619e21049f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:01.533800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:01.539695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:01.541552 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 891/1778] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:00 2022] DELETE /identity/v3/projects/cd2adc567771483baf43fc619e21049f => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:01.546873 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:01.549103 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:01.574066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19feeab-57ed-4a8c-9a67-eb668c00e3e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:01.574746 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19feeab-57ed-4a8c-9a67-eb668c00e3e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:01.575087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19feeab-57ed-4a8c-9a67-eb668c00e3e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:01.575368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19feeab-57ed-4a8c-9a67-eb668c00e3e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:01.577503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b19feeab-57ed-4a8c-9a67-eb668c00e3e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:01.670679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542ffdf-96b1-4ff0-ac72-6b3b12068306 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:01.686828 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3c6f8-c951-4f76-bbcf-73006412ac86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:01.688007 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3c6f8-c951-4f76-bbcf-73006412ac86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:01.689102 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3c6f8-c951-4f76-bbcf-73006412ac86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:01.690120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3c6f8-c951-4f76-bbcf-73006412ac86 admin admin] PATH_INFO: `/v3/users/031528b314c749dcb9961cfaad302909` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:01.708179 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c3c6f8-c951-4f76-bbcf-73006412ac86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=031528b314c749dcb9961cfaad302909)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:01.725528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c3c6f8-c951-4f76-bbcf-73006412ac86 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:01.765326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19feeab-57ed-4a8c-9a67-eb668c00e3e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:01.768373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19feeab-57ed-4a8c-9a67-eb668c00e3e1 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:01.791819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 888/1779] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:02.241048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-29c3c6f8-c951-4f76-bbcf-73006412ac86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '031528b314c749dcb9961cfaad302909'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:02.246178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:02.263461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-29c3c6f8-c951-4f76-bbcf-73006412ac86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '031528b314c749dcb9961cfaad302909'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:02.379705 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-29c3c6f8-c951-4f76-bbcf-73006412ac86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '031528b314c749dcb9961cfaad302909'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:02.379705 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-29c3c6f8-c951-4f76-bbcf-73006412ac86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '031528b314c749dcb9961cfaad302909'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:02.391746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-29c3c6f8-c951-4f76-bbcf-73006412ac86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '031528b314c749dcb9961cfaad302909'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:02.402593 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 892/1780] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:01 2022] DELETE /identity/v3/users/031528b314c749dcb9961cfaad302909 => generated 0 bytes in 853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:02.765831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19feeab-57ed-4a8c-9a67-eb668c00e3e1 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:02.765831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b19feeab-57ed-4a8c-9a67-eb668c00e3e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:02.868554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19feeab-57ed-4a8c-9a67-eb668c00e3e1 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:02.931856 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e935c4c-d9d5-4c74-9921-d237cc95c204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:02.933530 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e935c4c-d9d5-4c74-9921-d237cc95c204 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:02.933993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e935c4c-d9d5-4c74-9921-d237cc95c204 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:02.934486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e935c4c-d9d5-4c74-9921-d237cc95c204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:02.937653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9e935c4c-d9d5-4c74-9921-d237cc95c204 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:02.956241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e935c4c-d9d5-4c74-9921-d237cc95c204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:02.957558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e935c4c-d9d5-4c74-9921-d237cc95c204 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:02.981279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 889/1781] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:03.049941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3c6f8-c951-4f76-bbcf-73006412ac86 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:03.051533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-29c3c6f8-c951-4f76-bbcf-73006412ac86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:03.192235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3c6f8-c951-4f76-bbcf-73006412ac86 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:03.197151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af30b5-a0f0-492b-9663-9a2cd780c97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:03.198123 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af30b5-a0f0-492b-9663-9a2cd780c97c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:03.198477 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af30b5-a0f0-492b-9663-9a2cd780c97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:03.198825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af30b5-a0f0-492b-9663-9a2cd780c97c admin admin] PATH_INFO: `/v3/projects/9d630080d58a44b6b9dca2eef708b5f3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:03.200111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af30b5-a0f0-492b-9663-9a2cd780c97c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d630080d58a44b6b9dca2eef708b5f3)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:03.202425 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af30b5-a0f0-492b-9663-9a2cd780c97c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:03.442499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-10af30b5-a0f0-492b-9663-9a2cd780c97c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d630080d58a44b6b9dca2eef708b5f3', 'request_id': 'req-10af30b5-a0f0-492b-9663-9a2cd780c97c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:03.455292 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e935c4c-d9d5-4c74-9921-d237cc95c204 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:03.459026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-10af30b5-a0f0-492b-9663-9a2cd780c97c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d630080d58a44b6b9dca2eef708b5f3', 'request_id': 'req-10af30b5-a0f0-492b-9663-9a2cd780c97c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:03.459752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9e935c4c-d9d5-4c74-9921-d237cc95c204 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:03.478501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e935c4c-d9d5-4c74-9921-d237cc95c204 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:03.544170 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-10af30b5-a0f0-492b-9663-9a2cd780c97c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d630080d58a44b6b9dca2eef708b5f3', 'request_id': 'req-10af30b5-a0f0-492b-9663-9a2cd780c97c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:03.557309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6193f-79ea-4182-ba29-6c99c7eaea0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:03.559523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6193f-79ea-4182-ba29-6c99c7eaea0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:03.560033 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6193f-79ea-4182-ba29-6c99c7eaea0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:03.560324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6193f-79ea-4182-ba29-6c99c7eaea0c admin admin] PATH_INFO: `/v3/users/5dd2e26041364f1fbf7e738d552b8415` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:03.561771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6193f-79ea-4182-ba29-6c99c7eaea0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dd2e26041364f1fbf7e738d552b8415)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:03.563751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-10af30b5-a0f0-492b-9663-9a2cd780c97c admin admin] The token cache is being invalidate because project 9d630080d58a44b6b9dca2eef708b5f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:03.563886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6193f-79ea-4182-ba29-6c99c7eaea0c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:03.564231 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-10af30b5-a0f0-492b-9663-9a2cd780c97c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:03.566685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-10af30b5-a0f0-492b-9663-9a2cd780c97c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:03.568381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 893/1782] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:03 2022] DELETE /identity/v3/projects/9d630080d58a44b6b9dca2eef708b5f3 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:03.573637 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6ed16-c696-48ff-ab13-c9c270d15b05 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:03.576025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6ed16-c696-48ff-ab13-c9c270d15b05 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:03.576025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6ed16-c696-48ff-ab13-c9c270d15b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:03.606108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b4d6ed16-c696-48ff-ab13-c9c270d15b05 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:03.607769 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b4d6ed16-c696-48ff-ab13-c9c270d15b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:03.624238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 894/1783] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:03.627608 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75215c02-8c18-434f-8cb8-bb555eb92b73 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:03.628070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75215c02-8c18-434f-8cb8-bb555eb92b73 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:03.628495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75215c02-8c18-434f-8cb8-bb555eb92b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:03.661085 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-75215c02-8c18-434f-8cb8-bb555eb92b73 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:03.662498 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-75215c02-8c18-434f-8cb8-bb555eb92b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:03.671172 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 895/1784] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:03.675478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75215c02-8c18-434f-8cb8-bb555eb92b73 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:03.689169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75215c02-8c18-434f-8cb8-bb555eb92b73 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:03.693016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514b65d-767e-44ee-8673-6394923078d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:03.693839 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b65d-767e-44ee-8673-6394923078d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:03.694230 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b65d-767e-44ee-8673-6394923078d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:03.694633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b65d-767e-44ee-8673-6394923078d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:03.695491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514b65d-767e-44ee-8673-6394923078d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:03.695932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514b65d-767e-44ee-8673-6394923078d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:03.697835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514b65d-767e-44ee-8673-6394923078d3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:03.710379 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:03.712866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 896/1785] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:42:03.718044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514b65d-767e-44ee-8673-6394923078d3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:03.728438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514b65d-767e-44ee-8673-6394923078d3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:03.735126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7173615-9da7-4be1-9c3f-2a25ac20d799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:03.735802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7173615-9da7-4be1-9c3f-2a25ac20d799 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:03.736136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7173615-9da7-4be1-9c3f-2a25ac20d799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:03.736418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7173615-9da7-4be1-9c3f-2a25ac20d799 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:03.737345 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7173615-9da7-4be1-9c3f-2a25ac20d799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:03.739313 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7173615-9da7-4be1-9c3f-2a25ac20d799 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:03.799874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-bda6193f-79ea-4182-ba29-6c99c7eaea0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dd2e26041364f1fbf7e738d552b8415'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:03.800632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-bda6193f-79ea-4182-ba29-6c99c7eaea0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dd2e26041364f1fbf7e738d552b8415'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:03.801382 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-bda6193f-79ea-4182-ba29-6c99c7eaea0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dd2e26041364f1fbf7e738d552b8415'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:03.810016 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-bda6193f-79ea-4182-ba29-6c99c7eaea0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dd2e26041364f1fbf7e738d552b8415'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:03.826072 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-bda6193f-79ea-4182-ba29-6c99c7eaea0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dd2e26041364f1fbf7e738d552b8415'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:03.901389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 897/1786] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:03 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:03.907343 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7173615-9da7-4be1-9c3f-2a25ac20d799 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:03.909224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d7173615-9da7-4be1-9c3f-2a25ac20d799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:03.939405 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 890/1787] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:03 2022] DELETE /identity/v3/users/5dd2e26041364f1fbf7e738d552b8415 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:03.943914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957f6a1-d6e2-4bf7-a99b-22bdc073c676 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:03.944490 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957f6a1-d6e2-4bf7-a99b-22bdc073c676 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:03.944977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957f6a1-d6e2-4bf7-a99b-22bdc073c676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:03.967463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c957f6a1-d6e2-4bf7-a99b-22bdc073c676 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:03.970177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c957f6a1-d6e2-4bf7-a99b-22bdc073c676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:04.027075 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7173615-9da7-4be1-9c3f-2a25ac20d799 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:04.147833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 891/1788] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:04.156708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957f6a1-d6e2-4bf7-a99b-22bdc073c676 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:04.160254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c957f6a1-d6e2-4bf7-a99b-22bdc073c676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:04.274360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957f6a1-d6e2-4bf7-a99b-22bdc073c676 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:04.274596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad841a-bcac-4757-810a-d0f1db9ec755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:04.275355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad841a-bcac-4757-810a-d0f1db9ec755 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:04.275639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad841a-bcac-4757-810a-d0f1db9ec755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:04.276025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad841a-bcac-4757-810a-d0f1db9ec755 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:04.276730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad841a-bcac-4757-810a-d0f1db9ec755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:04.279203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad841a-bcac-4757-810a-d0f1db9ec755 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:04.290439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc151b0-db74-48d7-a74e-5af297ea3320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:04.293838 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc151b0-db74-48d7-a74e-5af297ea3320 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:04.296625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc151b0-db74-48d7-a74e-5af297ea3320 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:04.297095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc151b0-db74-48d7-a74e-5af297ea3320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:04.321891 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc151b0-db74-48d7-a74e-5af297ea3320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:04.323442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc151b0-db74-48d7-a74e-5af297ea3320 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:04.346327 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 892/1789] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:04.419290 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 898/1790] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:03 2022] POST /identity/v3/users => generated 293 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:04.421988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc151b0-db74-48d7-a74e-5af297ea3320 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:04.438587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc151b0-db74-48d7-a74e-5af297ea3320 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:04.445395 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149bddc-c5b2-4e47-9873-6509d80f9496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:04.446285 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149bddc-c5b2-4e47-9873-6509d80f9496 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:04.446723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149bddc-c5b2-4e47-9873-6509d80f9496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:04.447146 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149bddc-c5b2-4e47-9873-6509d80f9496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:04.448335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149bddc-c5b2-4e47-9873-6509d80f9496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:04.452744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149bddc-c5b2-4e47-9873-6509d80f9496 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:04.469009 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 893/1791] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:04.470321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad841a-bcac-4757-810a-d0f1db9ec755 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:04.488950 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad841a-bcac-4757-810a-d0f1db9ec755 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:04.494281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff0f5d-7c4f-4421-944d-d0b58aefab77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:04.495206 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff0f5d-7c4f-4421-944d-d0b58aefab77 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:04.495584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff0f5d-7c4f-4421-944d-d0b58aefab77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:04.496212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff0f5d-7c4f-4421-944d-d0b58aefab77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:04.497082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff0f5d-7c4f-4421-944d-d0b58aefab77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:04.499429 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff0f5d-7c4f-4421-944d-d0b58aefab77 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:04.523370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 899/1792] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:04.527262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149bddc-c5b2-4e47-9873-6509d80f9496 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:04.540125 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149bddc-c5b2-4e47-9873-6509d80f9496 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:04.559818 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968529e-6838-45b2-b523-53be3cf3a3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:04.565452 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968529e-6838-45b2-b523-53be3cf3a3a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:04.565452 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968529e-6838-45b2-b523-53be3cf3a3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:04.565452 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968529e-6838-45b2-b523-53be3cf3a3a9 admin admin] PATH_INFO: `/v3/projects/9b8dad9fc1d2487ab233ffbf7b8b993f/users/368a050c08fe44ceb662e69e307a7b47/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:04.579192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968529e-6838-45b2-b523-53be3cf3a3a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b8dad9fc1d2487ab233ffbf7b8b993f, user_id=368a050c08fe44ceb662e69e307a7b47, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:04.581938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968529e-6838-45b2-b523-53be3cf3a3a9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:04.748383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 894/1793] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:04 2022] PUT /identity/v3/projects/9b8dad9fc1d2487ab233ffbf7b8b993f/users/368a050c08fe44ceb662e69e307a7b47/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:04.758095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad3028-0e03-4815-82c6-747c253f91aa None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:04.758095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad3028-0e03-4815-82c6-747c253f91aa None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:04.758095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad3028-0e03-4815-82c6-747c253f91aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:04.799996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-28ad3028-0e03-4815-82c6-747c253f91aa None None] MFA Rules not processed for user `368a050c08fe44ceb662e69e307a7b47`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:04.802340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-28ad3028-0e03-4815-82c6-747c253f91aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:04.882260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 900/1794] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:04.886776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968529e-6838-45b2-b523-53be3cf3a3a9 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:04.964467 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968529e-6838-45b2-b523-53be3cf3a3a9 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:04.969475 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133083b-8083-4e4a-81dd-5eb93bb2c203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:04.970157 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133083b-8083-4e4a-81dd-5eb93bb2c203 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:04.970446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133083b-8083-4e4a-81dd-5eb93bb2c203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:04.970728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133083b-8083-4e4a-81dd-5eb93bb2c203 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:04.971440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133083b-8083-4e4a-81dd-5eb93bb2c203 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:04.973387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133083b-8083-4e4a-81dd-5eb93bb2c203 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:05.038851 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad3028-0e03-4815-82c6-747c253f91aa None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:05.038851 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-28ad3028-0e03-4815-82c6-747c253f91aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:05.048393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad3028-0e03-4815-82c6-747c253f91aa None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:05.053615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034683f6-7309-41e5-a05f-89cbca0a4ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:05.054576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034683f6-7309-41e5-a05f-89cbca0a4ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:05.054782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034683f6-7309-41e5-a05f-89cbca0a4ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:05.055066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034683f6-7309-41e5-a05f-89cbca0a4ad9 admin admin] PATH_INFO: `/v3/projects/a854dc79e12c435cbec1dc539e1414e9` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:05.056282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034683f6-7309-41e5-a05f-89cbca0a4ad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a854dc79e12c435cbec1dc539e1414e9)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:05.058298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034683f6-7309-41e5-a05f-89cbca0a4ad9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:05.134472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 895/1795] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:04 2022] POST /identity/v3/projects => generated 387 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:05.134472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1549ce-3acd-40f9-9c37-50009f226568 None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:05.134472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1549ce-3acd-40f9-9c37-50009f226568 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:05.134472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1549ce-3acd-40f9-9c37-50009f226568 None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:05.140914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 896/1796] 10.222.0.64 () {54 vars in 921 bytes} [Mon May 23 15:42:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:42:05.140914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1549ce-3acd-40f9-9c37-50009f226568 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:05.210476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1549ce-3acd-40f9-9c37-50009f226568 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:05.216450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae3877-1473-4a10-9e05-3f0447d6ddff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:05.217767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae3877-1473-4a10-9e05-3f0447d6ddff admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:05.218066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae3877-1473-4a10-9e05-3f0447d6ddff admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:05.218603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae3877-1473-4a10-9e05-3f0447d6ddff admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:05.219309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae3877-1473-4a10-9e05-3f0447d6ddff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:05.221440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae3877-1473-4a10-9e05-3f0447d6ddff admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:05.234708 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-034683f6-7309-41e5-a05f-89cbca0a4ad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a854dc79e12c435cbec1dc539e1414e9', 'request_id': 'req-034683f6-7309-41e5-a05f-89cbca0a4ad9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:05.241415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-034683f6-7309-41e5-a05f-89cbca0a4ad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a854dc79e12c435cbec1dc539e1414e9', 'request_id': 'req-034683f6-7309-41e5-a05f-89cbca0a4ad9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:05.312995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 897/1797] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:05 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:05.312995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd22e8-f7a8-4243-b40e-fc5f1ba90ea5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:05.312995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd22e8-f7a8-4243-b40e-fc5f1ba90ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:05.312995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd22e8-f7a8-4243-b40e-fc5f1ba90ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:05.331659 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-47cd22e8-f7a8-4243-b40e-fc5f1ba90ea5 None None] MFA Rules not processed for user `8da96bed73794fee8f719ec904936164`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:05.331939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-034683f6-7309-41e5-a05f-89cbca0a4ad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a854dc79e12c435cbec1dc539e1414e9', 'request_id': 'req-034683f6-7309-41e5-a05f-89cbca0a4ad9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:05.332874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-47cd22e8-f7a8-4243-b40e-fc5f1ba90ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:05.344868 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-034683f6-7309-41e5-a05f-89cbca0a4ad9 admin admin] The token cache is being invalidate because project a854dc79e12c435cbec1dc539e1414e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:05.344868 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-034683f6-7309-41e5-a05f-89cbca0a4ad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:05.345093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-034683f6-7309-41e5-a05f-89cbca0a4ad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:05.346582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 901/1798] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:05 2022] DELETE /identity/v3/projects/a854dc79e12c435cbec1dc539e1414e9 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:05.349521 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034683f6-7309-41e5-a05f-89cbca0a4ad9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:05.349521 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-034683f6-7309-41e5-a05f-89cbca0a4ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:05.399934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 898/1799] 10.222.0.64 () {58 vars in 1018 bytes} [Mon May 23 15:42:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:05.400744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd22e8-f7a8-4243-b40e-fc5f1ba90ea5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:05.402108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-47cd22e8-f7a8-4243-b40e-fc5f1ba90ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:05.413577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034683f6-7309-41e5-a05f-89cbca0a4ad9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:05.416186 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd22e8-f7a8-4243-b40e-fc5f1ba90ea5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:05.420895 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344bfa4-95d3-461b-a67a-f29517eb44e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:05.421601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344bfa4-95d3-461b-a67a-f29517eb44e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:05.421778 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbc83f-915f-4c3e-af3d-2b2415190a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:05.421894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344bfa4-95d3-461b-a67a-f29517eb44e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:05.422155 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344bfa4-95d3-461b-a67a-f29517eb44e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:05.422702 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbc83f-915f-4c3e-af3d-2b2415190a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:05.422882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344bfa4-95d3-461b-a67a-f29517eb44e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:05.422984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbc83f-915f-4c3e-af3d-2b2415190a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:05.423254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbc83f-915f-4c3e-af3d-2b2415190a9e admin admin] PATH_INFO: `/v3/users/9a8e6f332408432f9670f98c7cb39c67` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:05.424455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbc83f-915f-4c3e-af3d-2b2415190a9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8e6f332408432f9670f98c7cb39c67)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:05.424579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344bfa4-95d3-461b-a67a-f29517eb44e0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:05.426405 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbc83f-915f-4c3e-af3d-2b2415190a9e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:05.431615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:05.436825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 902/1800] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:05.440799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344bfa4-95d3-461b-a67a-f29517eb44e0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:05.442549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0344bfa4-95d3-461b-a67a-f29517eb44e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:05.458157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344bfa4-95d3-461b-a67a-f29517eb44e0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:05.463672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f01d02-c7ba-43ff-9a72-42b938c823e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:05.464325 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f01d02-c7ba-43ff-9a72-42b938c823e0 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:05.464620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f01d02-c7ba-43ff-9a72-42b938c823e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:05.464902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f01d02-c7ba-43ff-9a72-42b938c823e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:05.474396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f01d02-c7ba-43ff-9a72-42b938c823e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:05.476016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f01d02-c7ba-43ff-9a72-42b938c823e0 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:05.486759 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 903/1801] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:05.487547 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f01d02-c7ba-43ff-9a72-42b938c823e0 service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:05.497471 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f01d02-c7ba-43ff-9a72-42b938c823e0 service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:05.502613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0b6e3-7147-4c02-9c59-b5a4a4fddbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:05.507144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0b6e3-7147-4c02-9c59-b5a4a4fddbb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:05.507144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0b6e3-7147-4c02-9c59-b5a4a4fddbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:05.507144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0b6e3-7147-4c02-9c59-b5a4a4fddbb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:05.507144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0b6e3-7147-4c02-9c59-b5a4a4fddbb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:05.507144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0b6e3-7147-4c02-9c59-b5a4a4fddbb9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:05.516908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 904/1802] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:05.521003 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0b6e3-7147-4c02-9c59-b5a4a4fddbb9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:05.530937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0b6e3-7147-4c02-9c59-b5a4a4fddbb9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:05.535934 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5212b2-41f4-41a7-b692-0a0aaa4cae0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:05.536514 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5212b2-41f4-41a7-b692-0a0aaa4cae0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:05.536737 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5212b2-41f4-41a7-b692-0a0aaa4cae0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:05.537016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5212b2-41f4-41a7-b692-0a0aaa4cae0e admin admin] PATH_INFO: `/v3/projects/ff880b1e4d5c438d9ed81b8f10972aeb/users/94bedd383da2420db4ca8b0392d1a1c6/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:05.552416 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5212b2-41f4-41a7-b692-0a0aaa4cae0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff880b1e4d5c438d9ed81b8f10972aeb, user_id=94bedd383da2420db4ca8b0392d1a1c6, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:05.555245 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5212b2-41f4-41a7-b692-0a0aaa4cae0e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:05.660657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0ddbc83f-915f-4c3e-af3d-2b2415190a9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8e6f332408432f9670f98c7cb39c67'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:05.661308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0ddbc83f-915f-4c3e-af3d-2b2415190a9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8e6f332408432f9670f98c7cb39c67'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:05.661941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0ddbc83f-915f-4c3e-af3d-2b2415190a9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8e6f332408432f9670f98c7cb39c67'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:05.668851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0ddbc83f-915f-4c3e-af3d-2b2415190a9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8e6f332408432f9670f98c7cb39c67'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:05.681203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0ddbc83f-915f-4c3e-af3d-2b2415190a9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8e6f332408432f9670f98c7cb39c67'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:05.778649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 905/1803] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:05 2022] PUT /identity/v3/projects/ff880b1e4d5c438d9ed81b8f10972aeb/users/94bedd383da2420db4ca8b0392d1a1c6/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:05.790093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb969a7f-6dde-404a-8e99-614819c6c1e1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:05.790390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb969a7f-6dde-404a-8e99-614819c6c1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:05.790698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb969a7f-6dde-404a-8e99-614819c6c1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:05.819821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 899/1804] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:05 2022] DELETE /identity/v3/users/9a8e6f332408432f9670f98c7cb39c67 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:05.825245 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fb969a7f-6dde-404a-8e99-614819c6c1e1 None None] MFA Rules not processed for user `94bedd383da2420db4ca8b0392d1a1c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:05.826533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fb969a7f-6dde-404a-8e99-614819c6c1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:05.893537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 906/1805] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:05.898072 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbc83f-915f-4c3e-af3d-2b2415190a9e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:05.899675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0ddbc83f-915f-4c3e-af3d-2b2415190a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:05.965695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbc83f-915f-4c3e-af3d-2b2415190a9e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:05.970622 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2487b-fb07-436e-9f17-861bcf683903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:05.971295 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2487b-fb07-436e-9f17-861bcf683903 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:05.971588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2487b-fb07-436e-9f17-861bcf683903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:05.971875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2487b-fb07-436e-9f17-861bcf683903 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:05.972571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2487b-fb07-436e-9f17-861bcf683903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:05.974544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2487b-fb07-436e-9f17-861bcf683903 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:06.164900 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 900/1806] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:05 2022] POST /identity/v3/projects => generated 387 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:06.166339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb969a7f-6dde-404a-8e99-614819c6c1e1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:06.177109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb969a7f-6dde-404a-8e99-614819c6c1e1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:06.239229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d8e9b-8618-4924-b70f-4ef270829ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:06.239953 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d8e9b-8618-4924-b70f-4ef270829ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:06.240245 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d8e9b-8618-4924-b70f-4ef270829ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:06.240530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d8e9b-8618-4924-b70f-4ef270829ff8 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:06.241263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d8e9b-8618-4924-b70f-4ef270829ff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:06.243206 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d8e9b-8618-4924-b70f-4ef270829ff8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:06.362578 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 907/1807] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:06 2022] POST /identity/v3/users => generated 300 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:06.364631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2487b-fb07-436e-9f17-861bcf683903 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:06.366480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cec2487b-fb07-436e-9f17-861bcf683903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:06.375962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d8e9b-8618-4924-b70f-4ef270829ff8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:06.383009 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2487b-fb07-436e-9f17-861bcf683903 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:06.388464 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef65706-7b0d-474d-8fee-b78550caad7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:06.389540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef65706-7b0d-474d-8fee-b78550caad7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:06.389828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef65706-7b0d-474d-8fee-b78550caad7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:06.390106 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef65706-7b0d-474d-8fee-b78550caad7d admin admin] PATH_INFO: `/v3/projects/409b84a992cd492c8a2b3d17b15887c9` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:06.391350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef65706-7b0d-474d-8fee-b78550caad7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=409b84a992cd492c8a2b3d17b15887c9)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:06.393623 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef65706-7b0d-474d-8fee-b78550caad7d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:06.410060 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d8e9b-8618-4924-b70f-4ef270829ff8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:06.421106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071395f4-52d2-438a-bd7d-faae0f92fe16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:06.421770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071395f4-52d2-438a-bd7d-faae0f92fe16 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:06.422190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071395f4-52d2-438a-bd7d-faae0f92fe16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:06.422942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071395f4-52d2-438a-bd7d-faae0f92fe16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:06.423769 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071395f4-52d2-438a-bd7d-faae0f92fe16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:06.425682 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071395f4-52d2-438a-bd7d-faae0f92fe16 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:06.438406 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 908/1808] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:06.444726 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071395f4-52d2-438a-bd7d-faae0f92fe16 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:06.456757 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071395f4-52d2-438a-bd7d-faae0f92fe16 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:06.474598 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ba50d-bfbf-4356-b9e0-cdd4e31162aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:06.475645 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ba50d-bfbf-4356-b9e0-cdd4e31162aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:06.476215 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ba50d-bfbf-4356-b9e0-cdd4e31162aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:06.477305 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ba50d-bfbf-4356-b9e0-cdd4e31162aa admin admin] PATH_INFO: `/v3/projects/b846b65ddcd24d4eb2f4db2a63e509b0/users/7d0560b8f8ab4a338fb850ab833cee8b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:06.513099 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ba50d-bfbf-4356-b9e0-cdd4e31162aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b846b65ddcd24d4eb2f4db2a63e509b0, user_id=7d0560b8f8ab4a338fb850ab833cee8b, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:06.517595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ba50d-bfbf-4356-b9e0-cdd4e31162aa admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:06.638470 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 909/1809] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:06 2022] PUT /identity/v3/projects/b846b65ddcd24d4eb2f4db2a63e509b0/users/7d0560b8f8ab4a338fb850ab833cee8b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:06.642324 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ba50d-bfbf-4356-b9e0-cdd4e31162aa admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:06.707752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-eef65706-7b0d-474d-8fee-b78550caad7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '409b84a992cd492c8a2b3d17b15887c9', 'request_id': 'req-eef65706-7b0d-474d-8fee-b78550caad7d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:06.720313 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-eef65706-7b0d-474d-8fee-b78550caad7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '409b84a992cd492c8a2b3d17b15887c9', 'request_id': 'req-eef65706-7b0d-474d-8fee-b78550caad7d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:06.732300 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-eef65706-7b0d-474d-8fee-b78550caad7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '409b84a992cd492c8a2b3d17b15887c9', 'request_id': 'req-eef65706-7b0d-474d-8fee-b78550caad7d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:06.759594 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ba50d-bfbf-4356-b9e0-cdd4e31162aa admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:06.836506 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-eef65706-7b0d-474d-8fee-b78550caad7d admin admin] The token cache is being invalidate because project 409b84a992cd492c8a2b3d17b15887c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:06.837107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-eef65706-7b0d-474d-8fee-b78550caad7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:06.838507 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c611622-7f6e-459d-bf8c-75804aa3dfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:06.839652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c611622-7f6e-459d-bf8c-75804aa3dfce admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:06.839907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c611622-7f6e-459d-bf8c-75804aa3dfce admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:06.840332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c611622-7f6e-459d-bf8c-75804aa3dfce admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:06.840555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-eef65706-7b0d-474d-8fee-b78550caad7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:06.841389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c611622-7f6e-459d-bf8c-75804aa3dfce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:06.845618 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c611622-7f6e-459d-bf8c-75804aa3dfce admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:06.849948 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 901/1810] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:06 2022] DELETE /identity/v3/projects/409b84a992cd492c8a2b3d17b15887c9 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:06.853890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6852e4f-b452-4c95-9333-fd3f8e637be1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:06.854375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6852e4f-b452-4c95-9333-fd3f8e637be1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:06.854804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6852e4f-b452-4c95-9333-fd3f8e637be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:06.859458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 910/1811] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:06.863456 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c611622-7f6e-459d-bf8c-75804aa3dfce admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:06.866123 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9c611622-7f6e-459d-bf8c-75804aa3dfce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:06.885925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c611622-7f6e-459d-bf8c-75804aa3dfce admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:06.893496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7221bcb-1a90-4339-88ff-19e2aff86c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:06.894701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7221bcb-1a90-4339-88ff-19e2aff86c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:06.895119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7221bcb-1a90-4339-88ff-19e2aff86c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:06.895510 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7221bcb-1a90-4339-88ff-19e2aff86c22 admin admin] PATH_INFO: `/v3/projects/b846b65ddcd24d4eb2f4db2a63e509b0/users/7d0560b8f8ab4a338fb850ab833cee8b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:06.899119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7221bcb-1a90-4339-88ff-19e2aff86c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b846b65ddcd24d4eb2f4db2a63e509b0, user_id=7d0560b8f8ab4a338fb850ab833cee8b, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:06.902806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7221bcb-1a90-4339-88ff-19e2aff86c22 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:06.919559 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b6852e4f-b452-4c95-9333-fd3f8e637be1 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:06.922419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 911/1812] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:06 2022] PUT /identity/v3/projects/b846b65ddcd24d4eb2f4db2a63e509b0/users/7d0560b8f8ab4a338fb850ab833cee8b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:06.928250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b6852e4f-b452-4c95-9333-fd3f8e637be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:06.938012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e45d0-58f3-4533-b2b5-45ab07084d1f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:06.940070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e45d0-58f3-4533-b2b5-45ab07084d1f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:06.940517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e45d0-58f3-4533-b2b5-45ab07084d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:07.012834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d14e45d0-58f3-4533-b2b5-45ab07084d1f None None] MFA Rules not processed for user `7d0560b8f8ab4a338fb850ab833cee8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:07.014407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d14e45d0-58f3-4533-b2b5-45ab07084d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:07.025608 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:07.078565 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 902/1813] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:07.083683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6254e40-5932-4536-8416-e76a42d10a54 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:07.085975 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6254e40-5932-4536-8416-e76a42d10a54 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:07.086389 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6254e40-5932-4536-8416-e76a42d10a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:07.118581 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f6254e40-5932-4536-8416-e76a42d10a54 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:07.121231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f6254e40-5932-4536-8416-e76a42d10a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:07.133634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 903/1814] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:07.137816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6254e40-5932-4536-8416-e76a42d10a54 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:07.150492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6254e40-5932-4536-8416-e76a42d10a54 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:07.160380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591de84b-d8c7-4170-a7a9-776c1ef8f358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:07.160380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591de84b-d8c7-4170-a7a9-776c1ef8f358 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:07.161110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591de84b-d8c7-4170-a7a9-776c1ef8f358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:07.163455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 912/1815] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:07.165546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591de84b-d8c7-4170-a7a9-776c1ef8f358 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:07.166614 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591de84b-d8c7-4170-a7a9-776c1ef8f358 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:07.167062 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591de84b-d8c7-4170-a7a9-776c1ef8f358 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:07.169476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591de84b-d8c7-4170-a7a9-776c1ef8f358 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:07.175094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e45d0-58f3-4533-b2b5-45ab07084d1f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:07.176774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d14e45d0-58f3-4533-b2b5-45ab07084d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:07.208995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 904/1816] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:42:07.210354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591de84b-d8c7-4170-a7a9-776c1ef8f358 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:07.242303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591de84b-d8c7-4170-a7a9-776c1ef8f358 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:07.265738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d3e75-540f-424c-bd9a-f1a0d4d6edbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:07.266789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d3e75-540f-424c-bd9a-f1a0d4d6edbe admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:07.267342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d3e75-540f-424c-bd9a-f1a0d4d6edbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:07.267767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d3e75-540f-424c-bd9a-f1a0d4d6edbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:07.269299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d3e75-540f-424c-bd9a-f1a0d4d6edbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:07.277431 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d3e75-540f-424c-bd9a-f1a0d4d6edbe admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:07.278896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e45d0-58f3-4533-b2b5-45ab07084d1f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:07.297050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a89891-4e43-4477-9522-6e9578c03dbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:07.297050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a89891-4e43-4477-9522-6e9578c03dbb service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:07.297050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a89891-4e43-4477-9522-6e9578c03dbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:07.297050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a89891-4e43-4477-9522-6e9578c03dbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:07.321710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a89891-4e43-4477-9522-6e9578c03dbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:07.323261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a89891-4e43-4477-9522-6e9578c03dbb service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:07.338523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 913/1817] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:07.430700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 905/1818] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:07 2022] POST /identity/v3/projects => generated 361 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:07.438484 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a89891-4e43-4477-9522-6e9578c03dbb service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:07.458774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a89891-4e43-4477-9522-6e9578c03dbb service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:07.551873 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325ba35-dde7-4843-a2cb-b825986fdb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:07.552712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325ba35-dde7-4843-a2cb-b825986fdb82 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:07.553148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325ba35-dde7-4843-a2cb-b825986fdb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:07.553641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325ba35-dde7-4843-a2cb-b825986fdb82 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:07.554397 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325ba35-dde7-4843-a2cb-b825986fdb82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:07.557850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325ba35-dde7-4843-a2cb-b825986fdb82 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:07.686104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 914/1819] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:07 2022] POST /identity/v3/users => generated 281 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:07.688038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d3e75-540f-424c-bd9a-f1a0d4d6edbe admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:07.704767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d3e75-540f-424c-bd9a-f1a0d4d6edbe admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:07.719665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52148311-0b75-46ec-88ee-ea73ea728d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:07.720429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52148311-0b75-46ec-88ee-ea73ea728d18 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:07.720715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52148311-0b75-46ec-88ee-ea73ea728d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:07.721001 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52148311-0b75-46ec-88ee-ea73ea728d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:07.721843 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52148311-0b75-46ec-88ee-ea73ea728d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:07.723479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52148311-0b75-46ec-88ee-ea73ea728d18 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:07.755460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 906/1820] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:07.762087 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325ba35-dde7-4843-a2cb-b825986fdb82 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:07.782750 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325ba35-dde7-4843-a2cb-b825986fdb82 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:07.787505 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff708141-87b5-456f-a80b-9b4a6ff1cd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:07.788200 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff708141-87b5-456f-a80b-9b4a6ff1cd26 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:07.788549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff708141-87b5-456f-a80b-9b4a6ff1cd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:07.788835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff708141-87b5-456f-a80b-9b4a6ff1cd26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:07.789631 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff708141-87b5-456f-a80b-9b4a6ff1cd26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:07.791250 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff708141-87b5-456f-a80b-9b4a6ff1cd26 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:07.804926 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 915/1821] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:07.808713 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52148311-0b75-46ec-88ee-ea73ea728d18 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:07.820848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52148311-0b75-46ec-88ee-ea73ea728d18 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:07.826827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48379ac-4da4-48dc-b01c-3d8ac67c3de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:07.828461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48379ac-4da4-48dc-b01c-3d8ac67c3de1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:07.828748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48379ac-4da4-48dc-b01c-3d8ac67c3de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:07.829036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48379ac-4da4-48dc-b01c-3d8ac67c3de1 admin admin] PATH_INFO: `/v3/projects/37d22f4e61d7498987d48146f38f443f/users/ed18818b6b644896adf9ee529b20f5b8/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:07.852415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48379ac-4da4-48dc-b01c-3d8ac67c3de1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37d22f4e61d7498987d48146f38f443f, user_id=ed18818b6b644896adf9ee529b20f5b8, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:07.855288 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48379ac-4da4-48dc-b01c-3d8ac67c3de1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:07.907509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a18043-a4f7-437a-969e-467f58a31cfa None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:07.907509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a18043-a4f7-437a-969e-467f58a31cfa None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:07.907509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a18043-a4f7-437a-969e-467f58a31cfa None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:07.907509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 916/1822] 10.222.0.64 () {54 vars in 921 bytes} [Mon May 23 15:42:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:42:07.932857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365df42-db81-42a8-b741-3554eeedc86e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:07.933411 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365df42-db81-42a8-b741-3554eeedc86e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:07.933797 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365df42-db81-42a8-b741-3554eeedc86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:07.976009 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9365df42-db81-42a8-b741-3554eeedc86e None None] MFA Rules not processed for user `8da96bed73794fee8f719ec904936164`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:07.977744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9365df42-db81-42a8-b741-3554eeedc86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:07.986468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 907/1823] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:07 2022] PUT /identity/v3/projects/37d22f4e61d7498987d48146f38f443f/users/ed18818b6b644896adf9ee529b20f5b8/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:08.005962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d37d2c-9e55-4260-96e9-b7125649c00b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:08.006863 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d37d2c-9e55-4260-96e9-b7125649c00b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:08.007394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d37d2c-9e55-4260-96e9-b7125649c00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:08.057005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d4d37d2c-9e55-4260-96e9-b7125649c00b None None] MFA Rules not processed for user `ed18818b6b644896adf9ee529b20f5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:08.058788 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d4d37d2c-9e55-4260-96e9-b7125649c00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:08.081461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 917/1824] 10.222.0.64 () {58 vars in 1018 bytes} [Mon May 23 15:42:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:08.156227 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 908/1825] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:08.160558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365df42-db81-42a8-b741-3554eeedc86e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:08.189442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d37d2c-9e55-4260-96e9-b7125649c00b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:08.191061 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d4d37d2c-9e55-4260-96e9-b7125649c00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:08.227786 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d37d2c-9e55-4260-96e9-b7125649c00b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:08.251608 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d092d24-11d0-481b-8fa8-95cfbc966d71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:08.252291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d092d24-11d0-481b-8fa8-95cfbc966d71 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:08.252580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d092d24-11d0-481b-8fa8-95cfbc966d71 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:08.252963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d092d24-11d0-481b-8fa8-95cfbc966d71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:08.264124 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d092d24-11d0-481b-8fa8-95cfbc966d71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:08.266040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d092d24-11d0-481b-8fa8-95cfbc966d71 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:08.288609 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 909/1826] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:08.312099 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365df42-db81-42a8-b741-3554eeedc86e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:08.318372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bd1cd-bcd0-4064-bfcf-fa128ef63d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:08.319044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bd1cd-bcd0-4064-bfcf-fa128ef63d09 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:08.319339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bd1cd-bcd0-4064-bfcf-fa128ef63d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:08.319620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bd1cd-bcd0-4064-bfcf-fa128ef63d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:08.324983 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bd1cd-bcd0-4064-bfcf-fa128ef63d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:08.328912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bd1cd-bcd0-4064-bfcf-fa128ef63d09 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:08.732709 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:08.744470 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 918/1827] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:08 2022] POST /identity/v3/projects => generated 365 bytes in 584 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:08.747928 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d092d24-11d0-481b-8fa8-95cfbc966d71 service nova] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:08.760780 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d092d24-11d0-481b-8fa8-95cfbc966d71 service nova] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:08.833040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187941e-9004-43b1-b430-10eb74d96feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:08.834093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187941e-9004-43b1-b430-10eb74d96feb admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:08.834459 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187941e-9004-43b1-b430-10eb74d96feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:08.834844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187941e-9004-43b1-b430-10eb74d96feb admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:08.835638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187941e-9004-43b1-b430-10eb74d96feb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:08.837832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187941e-9004-43b1-b430-10eb74d96feb admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:09.039150 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 910/1828] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:08 2022] POST /identity/v3/users => generated 289 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:09.043564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bd1cd-bcd0-4064-bfcf-fa128ef63d09 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:09.057109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bd1cd-bcd0-4064-bfcf-fa128ef63d09 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:09.062523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0feb5-8946-4d06-8944-0db1f1e6fed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:09.063208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0feb5-8946-4d06-8944-0db1f1e6fed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:09.066360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0feb5-8946-4d06-8944-0db1f1e6fed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:09.066360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0feb5-8946-4d06-8944-0db1f1e6fed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:09.066360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0feb5-8946-4d06-8944-0db1f1e6fed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:09.066360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0feb5-8946-4d06-8944-0db1f1e6fed3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:09.081107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 919/1829] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:09.082328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187941e-9004-43b1-b430-10eb74d96feb admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:09.092977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187941e-9004-43b1-b430-10eb74d96feb admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:09.098683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b4605-3b98-46a6-b1ed-8c169c22937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:09.099160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b4605-3b98-46a6-b1ed-8c169c22937c admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:09.099451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b4605-3b98-46a6-b1ed-8c169c22937c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:09.099734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b4605-3b98-46a6-b1ed-8c169c22937c admin admin] PATH_INFO: `/v3/projects/0efb92f12914487da74cfed3d230a1fb/users/ff01aafb213246f889a41ab8a66c064d/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:09.119951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b4605-3b98-46a6-b1ed-8c169c22937c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0efb92f12914487da74cfed3d230a1fb, user_id=ff01aafb213246f889a41ab8a66c064d, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:09.122774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b4605-3b98-46a6-b1ed-8c169c22937c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:09.232890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 911/1830] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:09 2022] PUT /identity/v3/projects/0efb92f12914487da74cfed3d230a1fb/users/ff01aafb213246f889a41ab8a66c064d/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:09.237266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0feb5-8946-4d06-8944-0db1f1e6fed3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:09.247468 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0feb5-8946-4d06-8944-0db1f1e6fed3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:09.309949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b3dc6-8251-4b15-9752-f73532dc5285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:09.311227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b3dc6-8251-4b15-9752-f73532dc5285 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:09.311503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b3dc6-8251-4b15-9752-f73532dc5285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:09.311781 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b3dc6-8251-4b15-9752-f73532dc5285 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:09.312499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b3dc6-8251-4b15-9752-f73532dc5285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:09.314260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b3dc6-8251-4b15-9752-f73532dc5285 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:09.327979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 920/1831] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:09.329536 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b4605-3b98-46a6-b1ed-8c169c22937c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:09.339763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b4605-3b98-46a6-b1ed-8c169c22937c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:09.344537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45824c1f-e626-424e-b7e4-cd8f822c834a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:09.345456 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45824c1f-e626-424e-b7e4-cd8f822c834a admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:09.345744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45824c1f-e626-424e-b7e4-cd8f822c834a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:09.346031 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45824c1f-e626-424e-b7e4-cd8f822c834a admin admin] PATH_INFO: `/v3/projects/0efb92f12914487da74cfed3d230a1fb/users/ff01aafb213246f889a41ab8a66c064d/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:09.348058 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45824c1f-e626-424e-b7e4-cd8f822c834a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0efb92f12914487da74cfed3d230a1fb, user_id=ff01aafb213246f889a41ab8a66c064d, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:09.350783 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45824c1f-e626-424e-b7e4-cd8f822c834a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:09.364254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 912/1832] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:09 2022] PUT /identity/v3/projects/0efb92f12914487da74cfed3d230a1fb/users/ff01aafb213246f889a41ab8a66c064d/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:09.375724 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcca4b-7f1b-4c1f-ba09-3e750d9e155c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:09.376093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcca4b-7f1b-4c1f-ba09-3e750d9e155c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:09.376432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcca4b-7f1b-4c1f-ba09-3e750d9e155c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:09.403829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b2dcca4b-7f1b-4c1f-ba09-3e750d9e155c None None] MFA Rules not processed for user `ff01aafb213246f889a41ab8a66c064d`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:09.405756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b2dcca4b-7f1b-4c1f-ba09-3e750d9e155c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:09.472583 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 921/1833] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:09 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:09.569673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45824c1f-e626-424e-b7e4-cd8f822c834a admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:09.625116 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45824c1f-e626-424e-b7e4-cd8f822c834a admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:09.628864 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1a353-181c-413c-9878-5b7d2c03f753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:09.629722 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1a353-181c-413c-9878-5b7d2c03f753 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:09.636451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1a353-181c-413c-9878-5b7d2c03f753 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:09.636451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1a353-181c-413c-9878-5b7d2c03f753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:09.640962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1a353-181c-413c-9878-5b7d2c03f753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:09.642517 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1a353-181c-413c-9878-5b7d2c03f753 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:09.652262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 913/1834] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:09 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:10.098936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:10.287030 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcca4b-7f1b-4c1f-ba09-3e750d9e155c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:10.299156 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcca4b-7f1b-4c1f-ba09-3e750d9e155c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:10.303420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674c308-994e-45b8-9cc9-51f7176bc7db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:10.304265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674c308-994e-45b8-9cc9-51f7176bc7db service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:10.304701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674c308-994e-45b8-9cc9-51f7176bc7db service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:10.305813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674c308-994e-45b8-9cc9-51f7176bc7db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:10.307633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f674c308-994e-45b8-9cc9-51f7176bc7db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:10.382472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1a353-181c-413c-9878-5b7d2c03f753 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:10.383990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6eb1a353-181c-413c-9878-5b7d2c03f753 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:10.400111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674c308-994e-45b8-9cc9-51f7176bc7db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:10.400111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674c308-994e-45b8-9cc9-51f7176bc7db service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:10.400551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1a353-181c-413c-9878-5b7d2c03f753 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:10.405001 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 922/1835] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:10.407631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05af12-702c-4e71-9fb6-0c926076bd50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:10.408354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05af12-702c-4e71-9fb6-0c926076bd50 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:10.408589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05af12-702c-4e71-9fb6-0c926076bd50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:10.409011 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05af12-702c-4e71-9fb6-0c926076bd50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:10.502857 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05af12-702c-4e71-9fb6-0c926076bd50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:10.504478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05af12-702c-4e71-9fb6-0c926076bd50 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:10.517342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 914/1836] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:12.017826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:12.500122 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:13.018415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:13.545153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:14.487456 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674c308-994e-45b8-9cc9-51f7176bc7db service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:14.509467 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05af12-702c-4e71-9fb6-0c926076bd50 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:14.516040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1a05af12-702c-4e71-9fb6-0c926076bd50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:14.611485 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05af12-702c-4e71-9fb6-0c926076bd50 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:14.611994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674c308-994e-45b8-9cc9-51f7176bc7db service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:14.617474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a70523-d33d-40e4-bb02-92c831059fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:14.617922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6042b02-b6f7-47a8-a678-e0a5ab49b3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:14.618476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a70523-d33d-40e4-bb02-92c831059fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:14.618765 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a70523-d33d-40e4-bb02-92c831059fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:14.618870 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6042b02-b6f7-47a8-a678-e0a5ab49b3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:14.619037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a70523-d33d-40e4-bb02-92c831059fc4 admin admin] PATH_INFO: `/v3/users/ed18818b6b644896adf9ee529b20f5b8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:14.623815 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6042b02-b6f7-47a8-a678-e0a5ab49b3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:14.623815 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6042b02-b6f7-47a8-a678-e0a5ab49b3a1 admin admin] PATH_INFO: `/v3/users/3180f42b648243b7bc8aa0a26f024763` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:14.626429 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a70523-d33d-40e4-bb02-92c831059fc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed18818b6b644896adf9ee529b20f5b8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:14.626523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6042b02-b6f7-47a8-a678-e0a5ab49b3a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3180f42b648243b7bc8aa0a26f024763)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:14.626606 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a70523-d33d-40e4-bb02-92c831059fc4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:14.626716 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6042b02-b6f7-47a8-a678-e0a5ab49b3a1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:14.848862 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c7a70523-d33d-40e4-bb02-92c831059fc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed18818b6b644896adf9ee529b20f5b8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:14.871764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c7a70523-d33d-40e4-bb02-92c831059fc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed18818b6b644896adf9ee529b20f5b8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:14.885347 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e6042b02-b6f7-47a8-a678-e0a5ab49b3a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3180f42b648243b7bc8aa0a26f024763'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:14.886160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e6042b02-b6f7-47a8-a678-e0a5ab49b3a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3180f42b648243b7bc8aa0a26f024763'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:14.886840 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e6042b02-b6f7-47a8-a678-e0a5ab49b3a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3180f42b648243b7bc8aa0a26f024763'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:14.896993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e6042b02-b6f7-47a8-a678-e0a5ab49b3a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3180f42b648243b7bc8aa0a26f024763'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:14.911818 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e6042b02-b6f7-47a8-a678-e0a5ab49b3a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3180f42b648243b7bc8aa0a26f024763'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:14.968421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c7a70523-d33d-40e4-bb02-92c831059fc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed18818b6b644896adf9ee529b20f5b8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:14.969049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c7a70523-d33d-40e4-bb02-92c831059fc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed18818b6b644896adf9ee529b20f5b8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:14.980931 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c7a70523-d33d-40e4-bb02-92c831059fc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed18818b6b644896adf9ee529b20f5b8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:14.991421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 923/1837] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:14 2022] DELETE /identity/v3/users/ed18818b6b644896adf9ee529b20f5b8 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:14.996771 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e96cd-03f5-44ef-bcc1-5ff2a7e20b6c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:14.997098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e96cd-03f5-44ef-bcc1-5ff2a7e20b6c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:14.997348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e96cd-03f5-44ef-bcc1-5ff2a7e20b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:15.022369 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-061e96cd-03f5-44ef-bcc1-5ff2a7e20b6c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:15.034666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-061e96cd-03f5-44ef-bcc1-5ff2a7e20b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:15.083097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 915/1838] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:14 2022] DELETE /identity/v3/users/3180f42b648243b7bc8aa0a26f024763 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:15.084675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56d6e9-7a37-48a0-899d-0bac60e73d98 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:15.085099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56d6e9-7a37-48a0-899d-0bac60e73d98 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:15.085674 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56d6e9-7a37-48a0-899d-0bac60e73d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:15.136298 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:15.144799 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ca56d6e9-7a37-48a0-899d-0bac60e73d98 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:15.146250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ca56d6e9-7a37-48a0-899d-0bac60e73d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:15.178011 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 924/1839] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:15.206538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e96cd-03f5-44ef-bcc1-5ff2a7e20b6c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:15.207975 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-061e96cd-03f5-44ef-bcc1-5ff2a7e20b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:15.258571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 916/1840] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:15.291624 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e96cd-03f5-44ef-bcc1-5ff2a7e20b6c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:15.298583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56d6e9-7a37-48a0-899d-0bac60e73d98 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:15.300287 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ca56d6e9-7a37-48a0-899d-0bac60e73d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:15.377429 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca15d9-1725-4ff8-aca7-39cbf884d913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:15.379225 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca15d9-1725-4ff8-aca7-39cbf884d913 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:15.380261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca15d9-1725-4ff8-aca7-39cbf884d913 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:15.381568 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca15d9-1725-4ff8-aca7-39cbf884d913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:15.407869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca15d9-1725-4ff8-aca7-39cbf884d913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:15.410566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca15d9-1725-4ff8-aca7-39cbf884d913 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:15.437488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 925/1841] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:15.438872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56d6e9-7a37-48a0-899d-0bac60e73d98 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:15.445919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95ffa3-81ce-4234-88bf-2242238595ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:15.447235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95ffa3-81ce-4234-88bf-2242238595ed service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:15.447933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95ffa3-81ce-4234-88bf-2242238595ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:15.448544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95ffa3-81ce-4234-88bf-2242238595ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:15.463325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95ffa3-81ce-4234-88bf-2242238595ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:15.465030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95ffa3-81ce-4234-88bf-2242238595ed service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:15.476870 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 917/1842] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:15.659837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:15.830033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde605d-f04d-443d-83b7-c24f4d076c4a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:15.830710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde605d-f04d-443d-83b7-c24f4d076c4a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:15.831194 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde605d-f04d-443d-83b7-c24f4d076c4a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:15.833614 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 926/1843] 10.222.0.64 () {54 vars in 921 bytes} [Mon May 23 15:42:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:42:15.858803 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26824bac-9c54-4bcd-88d7-e95d5959498a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:15.859989 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26824bac-9c54-4bcd-88d7-e95d5959498a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:15.860786 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26824bac-9c54-4bcd-88d7-e95d5959498a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:15.881856 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-26824bac-9c54-4bcd-88d7-e95d5959498a None None] MFA Rules not processed for user `8da96bed73794fee8f719ec904936164`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:15.883083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-26824bac-9c54-4bcd-88d7-e95d5959498a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:15.975560 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 918/1844] 10.222.0.64 () {58 vars in 1018 bytes} [Mon May 23 15:42:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:16.013146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde605d-f04d-443d-83b7-c24f4d076c4a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:16.015248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5cde605d-f04d-443d-83b7-c24f4d076c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:16.036034 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde605d-f04d-443d-83b7-c24f4d076c4a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:16.042301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8461a-07fc-459c-ba5f-f93322ebd557 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:16.042984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8461a-07fc-459c-ba5f-f93322ebd557 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:16.043273 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8461a-07fc-459c-ba5f-f93322ebd557 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:16.043902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8461a-07fc-459c-ba5f-f93322ebd557 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:16.055595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a8461a-07fc-459c-ba5f-f93322ebd557 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:16.060229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a8461a-07fc-459c-ba5f-f93322ebd557 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:16.102967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26824bac-9c54-4bcd-88d7-e95d5959498a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:16.123040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 927/1845] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:16.124924 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-26824bac-9c54-4bcd-88d7-e95d5959498a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:16.136263 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26824bac-9c54-4bcd-88d7-e95d5959498a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:16.158851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08198b-da55-4345-9856-1ae96bbb3f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:16.160912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08198b-da55-4345-9856-1ae96bbb3f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:16.165405 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08198b-da55-4345-9856-1ae96bbb3f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:16.165781 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08198b-da55-4345-9856-1ae96bbb3f87 admin admin] PATH_INFO: `/v3/projects/37d22f4e61d7498987d48146f38f443f` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:16.170533 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d08198b-da55-4345-9856-1ae96bbb3f87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37d22f4e61d7498987d48146f38f443f)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:16.175695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d08198b-da55-4345-9856-1ae96bbb3f87 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:16.401458 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3d08198b-da55-4345-9856-1ae96bbb3f87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37d22f4e61d7498987d48146f38f443f', 'request_id': 'req-3d08198b-da55-4345-9856-1ae96bbb3f87'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:16.408968 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3d08198b-da55-4345-9856-1ae96bbb3f87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37d22f4e61d7498987d48146f38f443f', 'request_id': 'req-3d08198b-da55-4345-9856-1ae96bbb3f87'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:16.416638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3d08198b-da55-4345-9856-1ae96bbb3f87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37d22f4e61d7498987d48146f38f443f', 'request_id': 'req-3d08198b-da55-4345-9856-1ae96bbb3f87'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:16.544487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3d08198b-da55-4345-9856-1ae96bbb3f87 admin admin] The token cache is being invalidate because project 37d22f4e61d7498987d48146f38f443f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:16.545590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3d08198b-da55-4345-9856-1ae96bbb3f87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:16.546306 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3d08198b-da55-4345-9856-1ae96bbb3f87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:16.548012 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 919/1846] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:16 2022] DELETE /identity/v3/projects/37d22f4e61d7498987d48146f38f443f => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:16.552246 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8461a-07fc-459c-ba5f-f93322ebd557 service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:16.563339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8461a-07fc-459c-ba5f-f93322ebd557 service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:16.628606 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08198b-da55-4345-9856-1ae96bbb3f87 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:16.630071 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc7901-e22a-4897-b41d-4ae71b43fc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:16.630548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3d08198b-da55-4345-9856-1ae96bbb3f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:16.631044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc7901-e22a-4897-b41d-4ae71b43fc37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:16.631306 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc7901-e22a-4897-b41d-4ae71b43fc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:16.631545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc7901-e22a-4897-b41d-4ae71b43fc37 admin admin] PATH_INFO: `/v3/users/ff01aafb213246f889a41ab8a66c064d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:16.632768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc7901-e22a-4897-b41d-4ae71b43fc37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff01aafb213246f889a41ab8a66c064d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:16.634760 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc7901-e22a-4897-b41d-4ae71b43fc37 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:16.647506 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08198b-da55-4345-9856-1ae96bbb3f87 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:16.655841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efbba0-ff88-4536-b816-723d7a8e6304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:16.656607 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efbba0-ff88-4536-b816-723d7a8e6304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:16.656852 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efbba0-ff88-4536-b816-723d7a8e6304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:16.660025 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efbba0-ff88-4536-b816-723d7a8e6304 admin admin] PATH_INFO: `/v3/projects/9f964dc442d44b118374d6e44e13795c` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:16.662633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52efbba0-ff88-4536-b816-723d7a8e6304 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f964dc442d44b118374d6e44e13795c)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:16.665757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52efbba0-ff88-4536-b816-723d7a8e6304 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:16.993813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-02fc7901-e22a-4897-b41d-4ae71b43fc37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff01aafb213246f889a41ab8a66c064d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:16.997712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52efbba0-ff88-4536-b816-723d7a8e6304 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f964dc442d44b118374d6e44e13795c', 'request_id': 'req-52efbba0-ff88-4536-b816-723d7a8e6304'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:17.004169 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52efbba0-ff88-4536-b816-723d7a8e6304 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f964dc442d44b118374d6e44e13795c', 'request_id': 'req-52efbba0-ff88-4536-b816-723d7a8e6304'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:17.007472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-02fc7901-e22a-4897-b41d-4ae71b43fc37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff01aafb213246f889a41ab8a66c064d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:17.010999 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52efbba0-ff88-4536-b816-723d7a8e6304 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f964dc442d44b118374d6e44e13795c', 'request_id': 'req-52efbba0-ff88-4536-b816-723d7a8e6304'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:17.122096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-02fc7901-e22a-4897-b41d-4ae71b43fc37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff01aafb213246f889a41ab8a66c064d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:17.122875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-02fc7901-e22a-4897-b41d-4ae71b43fc37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff01aafb213246f889a41ab8a66c064d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:17.144189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-02fc7901-e22a-4897-b41d-4ae71b43fc37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff01aafb213246f889a41ab8a66c064d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:17.159018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 928/1847] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:16 2022] DELETE /identity/v3/users/ff01aafb213246f889a41ab8a66c064d => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:17.254960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52efbba0-ff88-4536-b816-723d7a8e6304 admin admin] The token cache is being invalidate because project 9f964dc442d44b118374d6e44e13795c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:17.254960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52efbba0-ff88-4536-b816-723d7a8e6304 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:17.254960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52efbba0-ff88-4536-b816-723d7a8e6304 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:17.254960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 920/1848] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:16 2022] DELETE /identity/v3/projects/9f964dc442d44b118374d6e44e13795c => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:17.258602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc7901-e22a-4897-b41d-4ae71b43fc37 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:17.260460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-02fc7901-e22a-4897-b41d-4ae71b43fc37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:17.431975 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc7901-e22a-4897-b41d-4ae71b43fc37 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:17.441460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6012c0b-da08-4a87-adec-957056bdfbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:17.442377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6012c0b-da08-4a87-adec-957056bdfbe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:17.442806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6012c0b-da08-4a87-adec-957056bdfbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:17.443096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6012c0b-da08-4a87-adec-957056bdfbe4 admin admin] PATH_INFO: `/v3/users/90a4bd8d252b4c50b43bc9d79cd3ece2` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:17.444333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6012c0b-da08-4a87-adec-957056bdfbe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90a4bd8d252b4c50b43bc9d79cd3ece2)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:17.446496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6012c0b-da08-4a87-adec-957056bdfbe4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:17.775412 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efbba0-ff88-4536-b816-723d7a8e6304 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:17.776922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-52efbba0-ff88-4536-b816-723d7a8e6304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:17.868523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efbba0-ff88-4536-b816-723d7a8e6304 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:17.879125 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07df04-d530-4676-a13e-80c628e42f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:17.879125 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07df04-d530-4676-a13e-80c628e42f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:17.879125 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07df04-d530-4676-a13e-80c628e42f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:17.879865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07df04-d530-4676-a13e-80c628e42f62 admin admin] PATH_INFO: `/v3/projects/0efb92f12914487da74cfed3d230a1fb` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:17.881305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07df04-d530-4676-a13e-80c628e42f62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0efb92f12914487da74cfed3d230a1fb)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:17.883575 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07df04-d530-4676-a13e-80c628e42f62 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:17.934448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c6012c0b-da08-4a87-adec-957056bdfbe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90a4bd8d252b4c50b43bc9d79cd3ece2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:17.948575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c6012c0b-da08-4a87-adec-957056bdfbe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90a4bd8d252b4c50b43bc9d79cd3ece2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:18.029196 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c6012c0b-da08-4a87-adec-957056bdfbe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90a4bd8d252b4c50b43bc9d79cd3ece2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:18.030591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c6012c0b-da08-4a87-adec-957056bdfbe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90a4bd8d252b4c50b43bc9d79cd3ece2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:18.039740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c6012c0b-da08-4a87-adec-957056bdfbe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90a4bd8d252b4c50b43bc9d79cd3ece2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:18.046667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 929/1849] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:17 2022] DELETE /identity/v3/users/90a4bd8d252b4c50b43bc9d79cd3ece2 => generated 0 bytes in 789 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:18.054076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c07df04-d530-4676-a13e-80c628e42f62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0efb92f12914487da74cfed3d230a1fb', 'request_id': 'req-9c07df04-d530-4676-a13e-80c628e42f62'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:18.060925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c07df04-d530-4676-a13e-80c628e42f62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0efb92f12914487da74cfed3d230a1fb', 'request_id': 'req-9c07df04-d530-4676-a13e-80c628e42f62'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:18.069886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c07df04-d530-4676-a13e-80c628e42f62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0efb92f12914487da74cfed3d230a1fb', 'request_id': 'req-9c07df04-d530-4676-a13e-80c628e42f62'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:18.226048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:18.240510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c07df04-d530-4676-a13e-80c628e42f62 admin admin] The token cache is being invalidate because project 0efb92f12914487da74cfed3d230a1fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:18.240974 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c07df04-d530-4676-a13e-80c628e42f62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:18.242825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c07df04-d530-4676-a13e-80c628e42f62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:18.249654 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 921/1850] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:17 2022] DELETE /identity/v3/projects/0efb92f12914487da74cfed3d230a1fb => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:18.251918 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128efb16-3c56-44f0-b8c0-15501925a7e0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:18.252465 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128efb16-3c56-44f0-b8c0-15501925a7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:18.252763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128efb16-3c56-44f0-b8c0-15501925a7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:18.277820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-128efb16-3c56-44f0-b8c0-15501925a7e0 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:18.279079 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-128efb16-3c56-44f0-b8c0-15501925a7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:18.347122 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 930/1851] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:18.355036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2705a1-dd59-416f-954d-60965c2199cc None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:18.355358 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2705a1-dd59-416f-954d-60965c2199cc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:18.355669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2705a1-dd59-416f-954d-60965c2199cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:18.384385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3d2705a1-dd59-416f-954d-60965c2199cc None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:18.384472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3d2705a1-dd59-416f-954d-60965c2199cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:18.397573 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 922/1852] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:18.399189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128efb16-3c56-44f0-b8c0-15501925a7e0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:18.409764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128efb16-3c56-44f0-b8c0-15501925a7e0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:18.414044 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:18.415465 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3855bcb-558d-4bfd-981d-5160742a47c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:18.416252 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3855bcb-558d-4bfd-981d-5160742a47c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:18.416539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3855bcb-558d-4bfd-981d-5160742a47c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:18.416816 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3855bcb-558d-4bfd-981d-5160742a47c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:18.417845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3855bcb-558d-4bfd-981d-5160742a47c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:18.429984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3855bcb-558d-4bfd-981d-5160742a47c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:18.429984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3855bcb-558d-4bfd-981d-5160742a47c9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:18.437135 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 931/1853] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:42:18.439648 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2705a1-dd59-416f-954d-60965c2199cc None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:18.454591 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2705a1-dd59-416f-954d-60965c2199cc None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:18.460952 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463bb7b-6d8d-448d-90ed-9a6373b12843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:18.462106 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463bb7b-6d8d-448d-90ed-9a6373b12843 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:18.463520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463bb7b-6d8d-448d-90ed-9a6373b12843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:18.464100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463bb7b-6d8d-448d-90ed-9a6373b12843 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:18.465023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463bb7b-6d8d-448d-90ed-9a6373b12843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:18.473179 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463bb7b-6d8d-448d-90ed-9a6373b12843 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:18.665251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3855bcb-558d-4bfd-981d-5160742a47c9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:18.668266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f3855bcb-558d-4bfd-981d-5160742a47c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:18.671930 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 923/1854] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:18 2022] POST /identity/v3/projects => generated 367 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:18.679653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463bb7b-6d8d-448d-90ed-9a6373b12843 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:18.691099 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3855bcb-558d-4bfd-981d-5160742a47c9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:18.770695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463bb7b-6d8d-448d-90ed-9a6373b12843 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:18.775659 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913796e-8bbd-470d-8436-71149e3a017f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:18.776375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913796e-8bbd-470d-8436-71149e3a017f admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:18.776670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913796e-8bbd-470d-8436-71149e3a017f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:18.776939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913796e-8bbd-470d-8436-71149e3a017f admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:18.777724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913796e-8bbd-470d-8436-71149e3a017f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:18.781107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913796e-8bbd-470d-8436-71149e3a017f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:18.781235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36393ade-61ef-4554-a54f-df9cfd6858f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:18.782124 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36393ade-61ef-4554-a54f-df9cfd6858f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:18.782459 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36393ade-61ef-4554-a54f-df9cfd6858f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:18.782734 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36393ade-61ef-4554-a54f-df9cfd6858f7 admin admin] PATH_INFO: `/v3/projects/108e7633965f465da1c2a0536f330c44` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:18.784615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36393ade-61ef-4554-a54f-df9cfd6858f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=108e7633965f465da1c2a0536f330c44)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:18.787215 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36393ade-61ef-4554-a54f-df9cfd6858f7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:18.887330 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 924/1855] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:18 2022] POST /identity/v3/users => generated 284 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:18.888321 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913796e-8bbd-470d-8436-71149e3a017f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:18.891335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3913796e-8bbd-470d-8436-71149e3a017f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:18.917206 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-36393ade-61ef-4554-a54f-df9cfd6858f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '108e7633965f465da1c2a0536f330c44', 'request_id': 'req-36393ade-61ef-4554-a54f-df9cfd6858f7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:18.925368 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-36393ade-61ef-4554-a54f-df9cfd6858f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '108e7633965f465da1c2a0536f330c44', 'request_id': 'req-36393ade-61ef-4554-a54f-df9cfd6858f7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:18.948903 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913796e-8bbd-470d-8436-71149e3a017f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:18.996548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa5df4-0f2b-4b76-b426-788e12554c56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:18.997247 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa5df4-0f2b-4b76-b426-788e12554c56 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:18.997538 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa5df4-0f2b-4b76-b426-788e12554c56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:18.997820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa5df4-0f2b-4b76-b426-788e12554c56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:19.000970 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-abaa5df4-0f2b-4b76-b426-788e12554c56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:19.001128 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-36393ade-61ef-4554-a54f-df9cfd6858f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '108e7633965f465da1c2a0536f330c44', 'request_id': 'req-36393ade-61ef-4554-a54f-df9cfd6858f7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:19.012344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-36393ade-61ef-4554-a54f-df9cfd6858f7 admin admin] The token cache is being invalidate because project 108e7633965f465da1c2a0536f330c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:19.012730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-36393ade-61ef-4554-a54f-df9cfd6858f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:19.013438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-36393ade-61ef-4554-a54f-df9cfd6858f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:19.015658 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 932/1856] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:18 2022] DELETE /identity/v3/projects/108e7633965f465da1c2a0536f330c44 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:19.016502 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36393ade-61ef-4554-a54f-df9cfd6858f7 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:19.017948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-36393ade-61ef-4554-a54f-df9cfd6858f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:19.062259 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa5df4-0f2b-4b76-b426-788e12554c56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:19.063801 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa5df4-0f2b-4b76-b426-788e12554c56 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:19.072727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 925/1857] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:19.074146 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223fefe-826c-49f4-b807-02d111110b12 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:19.074440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223fefe-826c-49f4-b807-02d111110b12 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:19.074798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223fefe-826c-49f4-b807-02d111110b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:19.083399 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36393ade-61ef-4554-a54f-df9cfd6858f7 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:19.089697 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7194b-f3c0-464b-9e59-3da65f93e337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:19.090371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7194b-f3c0-464b-9e59-3da65f93e337 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:19.090655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7194b-f3c0-464b-9e59-3da65f93e337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:19.091021 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7194b-f3c0-464b-9e59-3da65f93e337 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:19.091666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7194b-f3c0-464b-9e59-3da65f93e337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:19.096507 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d223fefe-826c-49f4-b807-02d111110b12 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:19.096630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7194b-f3c0-464b-9e59-3da65f93e337 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:19.096710 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d223fefe-826c-49f4-b807-02d111110b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:19.103066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 926/1858] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:19.106357 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 933/1859] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:19.108179 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223fefe-826c-49f4-b807-02d111110b12 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:19.111467 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b6174-bfe0-4a09-9532-06690cf4fdac None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:19.111767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b6174-bfe0-4a09-9532-06690cf4fdac None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:19.113776 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b6174-bfe0-4a09-9532-06690cf4fdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:19.127091 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223fefe-826c-49f4-b807-02d111110b12 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:19.134728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-074b6174-bfe0-4a09-9532-06690cf4fdac None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:19.135218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-074b6174-bfe0-4a09-9532-06690cf4fdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:19.141713 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a25d36-dbf8-41cd-a9e2-cb4f45a845c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:19.143496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a25d36-dbf8-41cd-a9e2-cb4f45a845c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:19.144426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a25d36-dbf8-41cd-a9e2-cb4f45a845c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:19.144851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a25d36-dbf8-41cd-a9e2-cb4f45a845c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:19.145972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a25d36-dbf8-41cd-a9e2-cb4f45a845c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:19.148839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a25d36-dbf8-41cd-a9e2-cb4f45a845c3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:19.150179 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 934/1860] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:19.151143 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b6174-bfe0-4a09-9532-06690cf4fdac None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:19.161911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b6174-bfe0-4a09-9532-06690cf4fdac None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:19.164900 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 927/1861] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:19.169381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a25d36-dbf8-41cd-a9e2-cb4f45a845c3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:19.178476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2d54a-304b-407c-9b0e-d9df74256b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:19.178476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2d54a-304b-407c-9b0e-d9df74256b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:19.178476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2d54a-304b-407c-9b0e-d9df74256b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:19.178476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2d54a-304b-407c-9b0e-d9df74256b1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:19.178798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2d54a-304b-407c-9b0e-d9df74256b1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:19.178798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2d54a-304b-407c-9b0e-d9df74256b1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:19.179875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2d54a-304b-407c-9b0e-d9df74256b1c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:19.190473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a25d36-dbf8-41cd-a9e2-cb4f45a845c3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:19.198043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 935/1862] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:42:19.204053 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2d54a-304b-407c-9b0e-d9df74256b1c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:19.204604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1572a-9819-4029-ab0c-13cbbd69c8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:19.206336 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1572a-9819-4029-ab0c-13cbbd69c8d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:19.206916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1572a-9819-4029-ab0c-13cbbd69c8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:19.207204 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1572a-9819-4029-ab0c-13cbbd69c8d0 admin admin] PATH_INFO: `/v3/projects/255c3a1ae37d483f8c21783f7922dfb7/users/b18f559b26f04bc3afdce16d7a10ac36/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:19.222559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2d54a-304b-407c-9b0e-d9df74256b1c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:19.224633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1572a-9819-4029-ab0c-13cbbd69c8d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=255c3a1ae37d483f8c21783f7922dfb7, user_id=b18f559b26f04bc3afdce16d7a10ac36, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:19.227920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1572a-9819-4029-ab0c-13cbbd69c8d0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:19.228062 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cde1e1-112b-4950-b8e9-7dbb3e303223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:19.228687 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cde1e1-112b-4950-b8e9-7dbb3e303223 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:19.228977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cde1e1-112b-4950-b8e9-7dbb3e303223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:19.229289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cde1e1-112b-4950-b8e9-7dbb3e303223 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:19.230218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cde1e1-112b-4950-b8e9-7dbb3e303223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:19.233689 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cde1e1-112b-4950-b8e9-7dbb3e303223 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:19.360189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 928/1863] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:19 2022] PUT /identity/v3/projects/255c3a1ae37d483f8c21783f7922dfb7/users/b18f559b26f04bc3afdce16d7a10ac36/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:19.361574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 936/1864] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:19 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:19.365504 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1572a-9819-4029-ab0c-13cbbd69c8d0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:19.371325 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a313755-4512-4c8c-bb35-07f55a0b648d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:19.371790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a313755-4512-4c8c-bb35-07f55a0b648d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:19.372173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a313755-4512-4c8c-bb35-07f55a0b648d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:19.410938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0a313755-4512-4c8c-bb35-07f55a0b648d None None] MFA Rules not processed for user `b18f559b26f04bc3afdce16d7a10ac36`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:19.413116 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0a313755-4512-4c8c-bb35-07f55a0b648d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:19.480582 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1572a-9819-4029-ab0c-13cbbd69c8d0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:19.505665 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 937/1865] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:19 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:19.506906 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a313755-4512-4c8c-bb35-07f55a0b648d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:19.568058 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e922c-a5b9-4daf-939e-0986104b5005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:19.569034 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e922c-a5b9-4daf-939e-0986104b5005 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:19.569517 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e922c-a5b9-4daf-939e-0986104b5005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:19.569871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e922c-a5b9-4daf-939e-0986104b5005 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:19.580328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e922c-a5b9-4daf-939e-0986104b5005 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:19.580328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e922c-a5b9-4daf-939e-0986104b5005 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:19.602389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a313755-4512-4c8c-bb35-07f55a0b648d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:19.607198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303ac05-a9c1-4ca2-a230-552f403b427d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:19.608005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303ac05-a9c1-4ca2-a230-552f403b427d admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:19.608286 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303ac05-a9c1-4ca2-a230-552f403b427d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:19.608680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303ac05-a9c1-4ca2-a230-552f403b427d admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:19.609409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303ac05-a9c1-4ca2-a230-552f403b427d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:19.611394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303ac05-a9c1-4ca2-a230-552f403b427d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:19.754601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 929/1866] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:19 2022] POST /identity/v3/users => generated 288 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:19.757664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e922c-a5b9-4daf-939e-0986104b5005 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:19.782520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e922c-a5b9-4daf-939e-0986104b5005 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:19.800498 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 938/1867] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:19 2022] POST /identity/v3/projects => generated 369 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:19.800705 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2780439-3cdb-4ebe-93a4-535f9e9a87b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:19.806250 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303ac05-a9c1-4ca2-a230-552f403b427d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:19.806484 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2780439-3cdb-4ebe-93a4-535f9e9a87b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:19.806647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2780439-3cdb-4ebe-93a4-535f9e9a87b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:19.808097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2780439-3cdb-4ebe-93a4-535f9e9a87b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:19.808894 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2780439-3cdb-4ebe-93a4-535f9e9a87b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:19.814515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2780439-3cdb-4ebe-93a4-535f9e9a87b5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:19.838983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 930/1868] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:19.840080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2780439-3cdb-4ebe-93a4-535f9e9a87b5 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:19.859735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2780439-3cdb-4ebe-93a4-535f9e9a87b5 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:19.899849 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303ac05-a9c1-4ca2-a230-552f403b427d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:19.905655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7292eb6-abb3-453b-ae63-cd0db468ec0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:19.906353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7292eb6-abb3-453b-ae63-cd0db468ec0b admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:19.906643 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7292eb6-abb3-453b-ae63-cd0db468ec0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:19.907185 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7292eb6-abb3-453b-ae63-cd0db468ec0b admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:19.907620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7292eb6-abb3-453b-ae63-cd0db468ec0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:19.909616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7292eb6-abb3-453b-ae63-cd0db468ec0b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:19.931415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f20ff-13fc-4356-a829-c5c15376307b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:19.932087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f20ff-13fc-4356-a829-c5c15376307b admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:19.932376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f20ff-13fc-4356-a829-c5c15376307b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:19.932689 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f20ff-13fc-4356-a829-c5c15376307b admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:19.933526 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f20ff-13fc-4356-a829-c5c15376307b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:19.935192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f20ff-13fc-4356-a829-c5c15376307b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:19.946726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 931/1869] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:19.951193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f20ff-13fc-4356-a829-c5c15376307b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:19.968807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f20ff-13fc-4356-a829-c5c15376307b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:19.973579 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff877a03-ba39-4267-8771-38cdc865b42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:19.974483 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff877a03-ba39-4267-8771-38cdc865b42b admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:19.975814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff877a03-ba39-4267-8771-38cdc865b42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:19.975814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff877a03-ba39-4267-8771-38cdc865b42b admin admin] PATH_INFO: `/v3/projects/ffba61b8eb664af5a7f968f90f9513bd/users/5a9411b008b34be6b948ede03e1acbfc/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:19.991511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff877a03-ba39-4267-8771-38cdc865b42b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffba61b8eb664af5a7f968f90f9513bd, user_id=5a9411b008b34be6b948ede03e1acbfc, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:19.994325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff877a03-ba39-4267-8771-38cdc865b42b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:20.059102 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 939/1870] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:19 2022] POST /identity/v3/users => generated 291 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:20.063458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7292eb6-abb3-453b-ae63-cd0db468ec0b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:20.073145 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7292eb6-abb3-453b-ae63-cd0db468ec0b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:20.078295 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f34f0-7ae0-41ea-8b75-a4d54096da58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:20.078964 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f34f0-7ae0-41ea-8b75-a4d54096da58 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:20.079258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f34f0-7ae0-41ea-8b75-a4d54096da58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:20.079544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f34f0-7ae0-41ea-8b75-a4d54096da58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:20.080276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f34f0-7ae0-41ea-8b75-a4d54096da58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:20.081937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f34f0-7ae0-41ea-8b75-a4d54096da58 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:20.092836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 940/1871] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:20.097669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f34f0-7ae0-41ea-8b75-a4d54096da58 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:20.101096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 932/1872] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:19 2022] PUT /identity/v3/projects/ffba61b8eb664af5a7f968f90f9513bd/users/5a9411b008b34be6b948ede03e1acbfc/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:20.112065 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b1cd7-6625-460c-b170-1d60e6b56410 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:20.112691 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b1cd7-6625-460c-b170-1d60e6b56410 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:20.113281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b1cd7-6625-460c-b170-1d60e6b56410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:20.117316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f34f0-7ae0-41ea-8b75-a4d54096da58 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:20.146589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d89b1cd7-6625-460c-b170-1d60e6b56410 None None] MFA Rules not processed for user `5a9411b008b34be6b948ede03e1acbfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:20.147954 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d89b1cd7-6625-460c-b170-1d60e6b56410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:20.187657 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58349189-1529-4260-ac2a-c09c1085db4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:20.188857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58349189-1529-4260-ac2a-c09c1085db4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:20.189177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58349189-1529-4260-ac2a-c09c1085db4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:20.189460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58349189-1529-4260-ac2a-c09c1085db4f admin admin] PATH_INFO: `/v3/projects/e487696322744408baa9bec8fdb0288a/users/b1b32d4908c8419ea80a2e92d91b92be/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:20.208771 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58349189-1529-4260-ac2a-c09c1085db4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e487696322744408baa9bec8fdb0288a, user_id=b1b32d4908c8419ea80a2e92d91b92be, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:20.211755 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58349189-1529-4260-ac2a-c09c1085db4f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:20.217149 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 933/1873] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:20.227587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b1cd7-6625-460c-b170-1d60e6b56410 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:20.228995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d89b1cd7-6625-460c-b170-1d60e6b56410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:20.285210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b1cd7-6625-460c-b170-1d60e6b56410 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:20.289052 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854e247-90b9-4230-9ae4-5c09197b0a2e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee7a6c342548f98e7dc73f4f71a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:20.289784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854e247-90b9-4230-9ae4-5c09197b0a2e service glance] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:20.290074 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854e247-90b9-4230-9ae4-5c09197b0a2e service glance] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:20.290351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854e247-90b9-4230-9ae4-5c09197b0a2e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:20.301172 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854e247-90b9-4230-9ae4-5c09197b0a2e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:20.302617 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854e247-90b9-4230-9ae4-5c09197b0a2e service glance] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:20.312784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 934/1874] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:20.392353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 941/1875] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:20 2022] PUT /identity/v3/projects/e487696322744408baa9bec8fdb0288a/users/b1b32d4908c8419ea80a2e92d91b92be/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:20.396104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854e247-90b9-4230-9ae4-5c09197b0a2e service glance] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:20.408966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854e247-90b9-4230-9ae4-5c09197b0a2e service glance] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:20.490724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a181ad-06af-4bae-acf5-41a34f698006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:20.490724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a181ad-06af-4bae-acf5-41a34f698006 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:20.490724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a181ad-06af-4bae-acf5-41a34f698006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:20.497220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a181ad-06af-4bae-acf5-41a34f698006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:20.499096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a181ad-06af-4bae-acf5-41a34f698006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:20.501306 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a181ad-06af-4bae-acf5-41a34f698006 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:20.515093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 935/1876] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:20.518130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58349189-1529-4260-ac2a-c09c1085db4f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:20.528470 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58349189-1529-4260-ac2a-c09c1085db4f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:20.543390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f277f-f22b-4eec-a2b3-1a28389afe2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:20.543390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f277f-f22b-4eec-a2b3-1a28389afe2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:20.543390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f277f-f22b-4eec-a2b3-1a28389afe2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:20.543390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f277f-f22b-4eec-a2b3-1a28389afe2e admin admin] PATH_INFO: `/v3/projects/e487696322744408baa9bec8fdb0288a/users/b1b32d4908c8419ea80a2e92d91b92be/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:20.547446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f277f-f22b-4eec-a2b3-1a28389afe2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e487696322744408baa9bec8fdb0288a, user_id=b1b32d4908c8419ea80a2e92d91b92be, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:20.550156 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f277f-f22b-4eec-a2b3-1a28389afe2e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:20.568936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 942/1877] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:20 2022] PUT /identity/v3/projects/e487696322744408baa9bec8fdb0288a/users/b1b32d4908c8419ea80a2e92d91b92be/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:20.583325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a111b-040a-48e2-a64a-a0b61badf4c0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:20.584887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a111b-040a-48e2-a64a-a0b61badf4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:20.585328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a111b-040a-48e2-a64a-a0b61badf4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:20.601939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:20.634225 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-5f4a111b-040a-48e2-a64a-a0b61badf4c0 None None] MFA Rules not processed for user `b1b32d4908c8419ea80a2e92d91b92be`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:20.635494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5f4a111b-040a-48e2-a64a-a0b61badf4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:20.719769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 936/1878] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:20 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:20.744387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:21.472384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f277f-f22b-4eec-a2b3-1a28389afe2e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:21.475042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dc7f277f-f22b-4eec-a2b3-1a28389afe2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:21.478090 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a111b-040a-48e2-a64a-a0b61badf4c0 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:21.479598 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5f4a111b-040a-48e2-a64a-a0b61badf4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:21.537041 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a111b-040a-48e2-a64a-a0b61badf4c0 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:21.539840 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f277f-f22b-4eec-a2b3-1a28389afe2e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:21.540642 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16139ba2-4624-4598-921e-3f686d4a6e8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:21.541335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16139ba2-4624-4598-921e-3f686d4a6e8a service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:21.541630 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16139ba2-4624-4598-921e-3f686d4a6e8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:21.541985 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16139ba2-4624-4598-921e-3f686d4a6e8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:21.545348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234893b-03d0-42d7-a184-90842c0b7cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:21.546220 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234893b-03d0-42d7-a184-90842c0b7cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:21.546505 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234893b-03d0-42d7-a184-90842c0b7cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:21.546779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234893b-03d0-42d7-a184-90842c0b7cf8 admin admin] PATH_INFO: `/v3/users/1bea05d85a4f4b0e9a0e730d8fd8c7a8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:21.547955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234893b-03d0-42d7-a184-90842c0b7cf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bea05d85a4f4b0e9a0e730d8fd8c7a8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:21.549951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234893b-03d0-42d7-a184-90842c0b7cf8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:21.552576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16139ba2-4624-4598-921e-3f686d4a6e8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:21.554163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16139ba2-4624-4598-921e-3f686d4a6e8a service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:21.564419 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 937/1879] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:21 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:21.893038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e234893b-03d0-42d7-a184-90842c0b7cf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bea05d85a4f4b0e9a0e730d8fd8c7a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:21.905221 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e234893b-03d0-42d7-a184-90842c0b7cf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bea05d85a4f4b0e9a0e730d8fd8c7a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:21.980873 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e234893b-03d0-42d7-a184-90842c0b7cf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bea05d85a4f4b0e9a0e730d8fd8c7a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:21.981549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e234893b-03d0-42d7-a184-90842c0b7cf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bea05d85a4f4b0e9a0e730d8fd8c7a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:22.005493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e234893b-03d0-42d7-a184-90842c0b7cf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bea05d85a4f4b0e9a0e730d8fd8c7a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:22.005493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 943/1880] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:21 2022] DELETE /identity/v3/users/1bea05d85a4f4b0e9a0e730d8fd8c7a8 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:22.006060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8496af-54ac-4df0-8206-41e828a91219 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:22.006060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8496af-54ac-4df0-8206-41e828a91219 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:22.006060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8496af-54ac-4df0-8206-41e828a91219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:22.035020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-de8496af-54ac-4df0-8206-41e828a91219 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:22.035926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-de8496af-54ac-4df0-8206-41e828a91219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:22.098715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 938/1881] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:22.108287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234893b-03d0-42d7-a184-90842c0b7cf8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:22.115320 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e234893b-03d0-42d7-a184-90842c0b7cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:22.198034 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234893b-03d0-42d7-a184-90842c0b7cf8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:22.202634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38d561-3e3a-41c0-a58a-345671baba74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:22.203445 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38d561-3e3a-41c0-a58a-345671baba74 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:22.203703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38d561-3e3a-41c0-a58a-345671baba74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:22.203990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38d561-3e3a-41c0-a58a-345671baba74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:22.215694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38d561-3e3a-41c0-a58a-345671baba74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:22.217999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38d561-3e3a-41c0-a58a-345671baba74 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:22.242369 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 944/1882] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:23.862516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8496af-54ac-4df0-8206-41e828a91219 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:23.863984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-de8496af-54ac-4df0-8206-41e828a91219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:23.877804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8496af-54ac-4df0-8206-41e828a91219 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:23.883108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a1a71-d39c-4d21-bedc-943a255b667b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:23.885271 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a1a71-d39c-4d21-bedc-943a255b667b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:23.885271 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a1a71-d39c-4d21-bedc-943a255b667b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:23.885271 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a1a71-d39c-4d21-bedc-943a255b667b admin admin] PATH_INFO: `/v3/projects/ca944beed5584d4f816e39cb1c6098d9` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:23.886120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a1a71-d39c-4d21-bedc-943a255b667b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca944beed5584d4f816e39cb1c6098d9)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:23.888128 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a1a71-d39c-4d21-bedc-943a255b667b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:24.145190 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c4a1a71-d39c-4d21-bedc-943a255b667b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca944beed5584d4f816e39cb1c6098d9', 'request_id': 'req-9c4a1a71-d39c-4d21-bedc-943a255b667b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:24.151362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c4a1a71-d39c-4d21-bedc-943a255b667b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca944beed5584d4f816e39cb1c6098d9', 'request_id': 'req-9c4a1a71-d39c-4d21-bedc-943a255b667b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:24.158251 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c4a1a71-d39c-4d21-bedc-943a255b667b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca944beed5584d4f816e39cb1c6098d9', 'request_id': 'req-9c4a1a71-d39c-4d21-bedc-943a255b667b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:24.316270 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c4a1a71-d39c-4d21-bedc-943a255b667b admin admin] The token cache is being invalidate because project ca944beed5584d4f816e39cb1c6098d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:24.316904 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c4a1a71-d39c-4d21-bedc-943a255b667b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:24.317667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9c4a1a71-d39c-4d21-bedc-943a255b667b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:24.319441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 939/1883] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:23 2022] DELETE /identity/v3/projects/ca944beed5584d4f816e39cb1c6098d9 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:24.323362 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38d561-3e3a-41c0-a58a-345671baba74 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:24.343189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38d561-3e3a-41c0-a58a-345671baba74 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:24.343189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:24.408768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84607fa5-9efa-4e33-accf-d04c75d8b145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:24.409733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84607fa5-9efa-4e33-accf-d04c75d8b145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:24.410025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84607fa5-9efa-4e33-accf-d04c75d8b145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:24.410311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84607fa5-9efa-4e33-accf-d04c75d8b145 admin admin] PATH_INFO: `/v3/users/d430d9949e1b446ba54c490ad445855b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:24.411540 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84607fa5-9efa-4e33-accf-d04c75d8b145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d430d9949e1b446ba54c490ad445855b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:24.413519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84607fa5-9efa-4e33-accf-d04c75d8b145 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:24.669333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-84607fa5-9efa-4e33-accf-d04c75d8b145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd430d9949e1b446ba54c490ad445855b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:24.682899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-84607fa5-9efa-4e33-accf-d04c75d8b145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd430d9949e1b446ba54c490ad445855b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:24.790432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-84607fa5-9efa-4e33-accf-d04c75d8b145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd430d9949e1b446ba54c490ad445855b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:24.791038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-84607fa5-9efa-4e33-accf-d04c75d8b145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd430d9949e1b446ba54c490ad445855b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:24.799471 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-84607fa5-9efa-4e33-accf-d04c75d8b145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd430d9949e1b446ba54c490ad445855b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:24.806198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 945/1884] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:24 2022] DELETE /identity/v3/users/d430d9949e1b446ba54c490ad445855b => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:25.243330 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a1a71-d39c-4d21-bedc-943a255b667b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:25.244814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9c4a1a71-d39c-4d21-bedc-943a255b667b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:25.275338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a1a71-d39c-4d21-bedc-943a255b667b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:25.360135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e241f-842a-4e3c-bfc5-76dae9b0496f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:25.361887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e241f-842a-4e3c-bfc5-76dae9b0496f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:25.362190 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e241f-842a-4e3c-bfc5-76dae9b0496f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:25.362477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e241f-842a-4e3c-bfc5-76dae9b0496f admin admin] PATH_INFO: `/v3/projects/0e18c4b9981342249f01c73ccd2f287f` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:25.363896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e241f-842a-4e3c-bfc5-76dae9b0496f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e18c4b9981342249f01c73ccd2f287f)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:25.366322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e241f-842a-4e3c-bfc5-76dae9b0496f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:25.367648 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:25.454359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:25.562825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6e4e241f-842a-4e3c-bfc5-76dae9b0496f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e18c4b9981342249f01c73ccd2f287f', 'request_id': 'req-6e4e241f-842a-4e3c-bfc5-76dae9b0496f'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:25.569147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6e4e241f-842a-4e3c-bfc5-76dae9b0496f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e18c4b9981342249f01c73ccd2f287f', 'request_id': 'req-6e4e241f-842a-4e3c-bfc5-76dae9b0496f'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:25.578698 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6e4e241f-842a-4e3c-bfc5-76dae9b0496f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e18c4b9981342249f01c73ccd2f287f', 'request_id': 'req-6e4e241f-842a-4e3c-bfc5-76dae9b0496f'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:25.693689 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6e4e241f-842a-4e3c-bfc5-76dae9b0496f admin admin] The token cache is being invalidate because project 0e18c4b9981342249f01c73ccd2f287f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:25.694235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6e4e241f-842a-4e3c-bfc5-76dae9b0496f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:25.695372 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6e4e241f-842a-4e3c-bfc5-76dae9b0496f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:25.697125 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 940/1885] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:25 2022] DELETE /identity/v3/projects/0e18c4b9981342249f01c73ccd2f287f => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:25.703782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab9a8e-27bf-498e-afd9-885bd534a26b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:25.704068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab9a8e-27bf-498e-afd9-885bd534a26b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:25.704342 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab9a8e-27bf-498e-afd9-885bd534a26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:25.726917 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e7ab9a8e-27bf-498e-afd9-885bd534a26b None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:25.728133 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e7ab9a8e-27bf-498e-afd9-885bd534a26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:25.791426 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 946/1886] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:25.800625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbfe06-d9f1-4bfa-9ab4-460fee72a463 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:25.800922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbfe06-d9f1-4bfa-9ab4-460fee72a463 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:25.801370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbfe06-d9f1-4bfa-9ab4-460fee72a463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:25.821253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-30bbfe06-d9f1-4bfa-9ab4-460fee72a463 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:25.822434 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-30bbfe06-d9f1-4bfa-9ab4-460fee72a463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:25.832067 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 941/1887] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:25.836315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab9a8e-27bf-498e-afd9-885bd534a26b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:25.846569 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab9a8e-27bf-498e-afd9-885bd534a26b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:25.851127 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ade1f8-04f0-4cfa-aff1-e5483cb6ab46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:25.851798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ade1f8-04f0-4cfa-aff1-e5483cb6ab46 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:25.852160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ade1f8-04f0-4cfa-aff1-e5483cb6ab46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:25.852456 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ade1f8-04f0-4cfa-aff1-e5483cb6ab46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:25.853223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ade1f8-04f0-4cfa-aff1-e5483cb6ab46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:25.853553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ade1f8-04f0-4cfa-aff1-e5483cb6ab46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:25.855153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ade1f8-04f0-4cfa-aff1-e5483cb6ab46 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:25.867433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 947/1888] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:42:25.871620 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbfe06-d9f1-4bfa-9ab4-460fee72a463 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:25.881442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbfe06-d9f1-4bfa-9ab4-460fee72a463 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:25.886190 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df353d4-dfc0-4991-94d1-afbb5c95ad37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:25.886854 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df353d4-dfc0-4991-94d1-afbb5c95ad37 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:25.887134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df353d4-dfc0-4991-94d1-afbb5c95ad37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:25.887408 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df353d4-dfc0-4991-94d1-afbb5c95ad37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:25.888096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df353d4-dfc0-4991-94d1-afbb5c95ad37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:25.890554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df353d4-dfc0-4991-94d1-afbb5c95ad37 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:25.983230 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 942/1889] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:25 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:25.987515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ade1f8-04f0-4cfa-aff1-e5483cb6ab46 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:25.997978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ade1f8-04f0-4cfa-aff1-e5483cb6ab46 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:26.055966 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3c752-d25f-4501-a8d9-c517c43a71f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:26.056821 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3c752-d25f-4501-a8d9-c517c43a71f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:26.057217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3c752-d25f-4501-a8d9-c517c43a71f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:26.057574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3c752-d25f-4501-a8d9-c517c43a71f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:26.058356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3c752-d25f-4501-a8d9-c517c43a71f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:26.060419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3c752-d25f-4501-a8d9-c517c43a71f9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:26.190327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 948/1890] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:25 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:26.195756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df353d4-dfc0-4991-94d1-afbb5c95ad37 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:26.209381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df353d4-dfc0-4991-94d1-afbb5c95ad37 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:26.217086 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdd30d-23cd-4e4e-835f-9afce3dd8d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:26.221534 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdd30d-23cd-4e4e-835f-9afce3dd8d94 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:26.221534 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdd30d-23cd-4e4e-835f-9afce3dd8d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:26.221534 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdd30d-23cd-4e4e-835f-9afce3dd8d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:26.221534 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdd30d-23cd-4e4e-835f-9afce3dd8d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:26.222257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdd30d-23cd-4e4e-835f-9afce3dd8d94 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:26.234243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 943/1891] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:26.241813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3c752-d25f-4501-a8d9-c517c43a71f9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:26.265396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3c752-d25f-4501-a8d9-c517c43a71f9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:26.265396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af0d2c-2d7a-4f4d-9449-62eca45793ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:26.265396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af0d2c-2d7a-4f4d-9449-62eca45793ac admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:26.265396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af0d2c-2d7a-4f4d-9449-62eca45793ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:26.265764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af0d2c-2d7a-4f4d-9449-62eca45793ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:26.265764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af0d2c-2d7a-4f4d-9449-62eca45793ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:26.265764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af0d2c-2d7a-4f4d-9449-62eca45793ac admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:26.277790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 949/1892] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:26.283147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdd30d-23cd-4e4e-835f-9afce3dd8d94 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:26.293189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdd30d-23cd-4e4e-835f-9afce3dd8d94 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:26.300432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90a216-001f-40f2-aca2-03b4bdaa204c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:26.301479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90a216-001f-40f2-aca2-03b4bdaa204c admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:26.301764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90a216-001f-40f2-aca2-03b4bdaa204c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:26.302044 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90a216-001f-40f2-aca2-03b4bdaa204c admin admin] PATH_INFO: `/v3/projects/4c9d7a6c555f4bf4960660292e368aa4/users/d91414e8c7da478bb8c73f42cac64e02/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:26.318395 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90a216-001f-40f2-aca2-03b4bdaa204c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c9d7a6c555f4bf4960660292e368aa4, user_id=d91414e8c7da478bb8c73f42cac64e02, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:26.321448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90a216-001f-40f2-aca2-03b4bdaa204c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:26.422654 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 944/1893] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:26 2022] PUT /identity/v3/projects/4c9d7a6c555f4bf4960660292e368aa4/users/d91414e8c7da478bb8c73f42cac64e02/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:26.432359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a3ec7-ddd0-4164-af53-9316865ea2ec None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:26.432873 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a3ec7-ddd0-4164-af53-9316865ea2ec None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:26.433130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a3ec7-ddd0-4164-af53-9316865ea2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:26.463744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-dd6a3ec7-ddd0-4164-af53-9316865ea2ec None None] MFA Rules not processed for user `d91414e8c7da478bb8c73f42cac64e02`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:26.466013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dd6a3ec7-ddd0-4164-af53-9316865ea2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:26.564468 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 950/1894] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:26.569142 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90a216-001f-40f2-aca2-03b4bdaa204c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:26.637887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90a216-001f-40f2-aca2-03b4bdaa204c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:26.644653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1413e-1fa0-416e-8271-bcf76181f1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:26.645472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1413e-1fa0-416e-8271-bcf76181f1c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:26.645784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1413e-1fa0-416e-8271-bcf76181f1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:26.646069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1413e-1fa0-416e-8271-bcf76181f1c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:26.646773 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1413e-1fa0-416e-8271-bcf76181f1c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:26.648690 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1413e-1fa0-416e-8271-bcf76181f1c8 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:26.744677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 945/1895] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:26 2022] POST /identity/v3/projects => generated 379 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:26.745561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a3ec7-ddd0-4164-af53-9316865ea2ec None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:26.758004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a3ec7-ddd0-4164-af53-9316865ea2ec None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:26.851202 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f45f6c-0920-4964-bfb2-0d86e325a033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:26.851890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f45f6c-0920-4964-bfb2-0d86e325a033 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:26.852180 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f45f6c-0920-4964-bfb2-0d86e325a033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:26.852462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f45f6c-0920-4964-bfb2-0d86e325a033 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:26.853253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f45f6c-0920-4964-bfb2-0d86e325a033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:26.855123 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f45f6c-0920-4964-bfb2-0d86e325a033 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:26.969705 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 951/1896] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:26 2022] POST /identity/v3/users => generated 290 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:26.971023 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f45f6c-0920-4964-bfb2-0d86e325a033 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:26.971228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1413e-1fa0-416e-8271-bcf76181f1c8 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:26.973519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6fd1413e-1fa0-416e-8271-bcf76181f1c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:26.982202 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f45f6c-0920-4964-bfb2-0d86e325a033 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:26.987726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1413e-1fa0-416e-8271-bcf76181f1c8 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:26.988536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c304fd6-c150-4d84-b868-82dc3c6cf704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:26.989491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c304fd6-c150-4d84-b868-82dc3c6cf704 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:26.989778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c304fd6-c150-4d84-b868-82dc3c6cf704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:26.990217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c304fd6-c150-4d84-b868-82dc3c6cf704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:26.991263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c304fd6-c150-4d84-b868-82dc3c6cf704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:26.993010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c304fd6-c150-4d84-b868-82dc3c6cf704 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:26.993532 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44eb5e7-b24c-44e4-9eda-724ef9fed390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:26.994508 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44eb5e7-b24c-44e4-9eda-724ef9fed390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:26.994939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44eb5e7-b24c-44e4-9eda-724ef9fed390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:26.995181 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44eb5e7-b24c-44e4-9eda-724ef9fed390 admin admin] PATH_INFO: `/v3/users/368a050c08fe44ceb662e69e307a7b47` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:26.996535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44eb5e7-b24c-44e4-9eda-724ef9fed390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=368a050c08fe44ceb662e69e307a7b47)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:26.999402 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44eb5e7-b24c-44e4-9eda-724ef9fed390 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:27.014164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 952/1897] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:27.020198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c304fd6-c150-4d84-b868-82dc3c6cf704 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:27.032481 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c304fd6-c150-4d84-b868-82dc3c6cf704 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:27.037911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38e2bb-da17-4de3-ac63-f449b79e3507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:27.038787 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38e2bb-da17-4de3-ac63-f449b79e3507 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:27.039231 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38e2bb-da17-4de3-ac63-f449b79e3507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:27.042178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38e2bb-da17-4de3-ac63-f449b79e3507 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:27.042178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38e2bb-da17-4de3-ac63-f449b79e3507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:27.042178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38e2bb-da17-4de3-ac63-f449b79e3507 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:27.055383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 953/1898] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:27.060038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38e2bb-da17-4de3-ac63-f449b79e3507 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:27.072728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38e2bb-da17-4de3-ac63-f449b79e3507 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:27.077840 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad960c-b542-406a-9776-15509baa0945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:27.079011 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad960c-b542-406a-9776-15509baa0945 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:27.079377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad960c-b542-406a-9776-15509baa0945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:27.079794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad960c-b542-406a-9776-15509baa0945 admin admin] PATH_INFO: `/v3/projects/a09b0ba895f94450b2fe5f3d3a358de7/users/20efc5c80a8446d5bd6507520b617036/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:27.098700 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad960c-b542-406a-9776-15509baa0945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a09b0ba895f94450b2fe5f3d3a358de7, user_id=20efc5c80a8446d5bd6507520b617036, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:27.101651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad960c-b542-406a-9776-15509baa0945 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:27.232486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 954/1899] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:27 2022] PUT /identity/v3/projects/a09b0ba895f94450b2fe5f3d3a358de7/users/20efc5c80a8446d5bd6507520b617036/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:27.238797 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c44eb5e7-b24c-44e4-9eda-724ef9fed390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '368a050c08fe44ceb662e69e307a7b47'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:27.239466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c44eb5e7-b24c-44e4-9eda-724ef9fed390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '368a050c08fe44ceb662e69e307a7b47'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:27.240079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c44eb5e7-b24c-44e4-9eda-724ef9fed390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '368a050c08fe44ceb662e69e307a7b47'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:27.246461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932fb69-fc49-4e3f-a290-5a17df0217c8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:27.246949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932fb69-fc49-4e3f-a290-5a17df0217c8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:27.247503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932fb69-fc49-4e3f-a290-5a17df0217c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:27.250089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c44eb5e7-b24c-44e4-9eda-724ef9fed390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '368a050c08fe44ceb662e69e307a7b47'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:27.263319 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c44eb5e7-b24c-44e4-9eda-724ef9fed390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '368a050c08fe44ceb662e69e307a7b47'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:27.284108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d932fb69-fc49-4e3f-a290-5a17df0217c8 None None] MFA Rules not processed for user `20efc5c80a8446d5bd6507520b617036`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:27.285490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d932fb69-fc49-4e3f-a290-5a17df0217c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:27.348314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 946/1900] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:26 2022] DELETE /identity/v3/users/368a050c08fe44ceb662e69e307a7b47 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:27.360766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166c0c9-66bf-44f7-b73a-c0b67ae57393 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:27.360978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166c0c9-66bf-44f7-b73a-c0b67ae57393 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:27.361265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166c0c9-66bf-44f7-b73a-c0b67ae57393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:27.388527 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f166c0c9-66bf-44f7-b73a-c0b67ae57393 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:27.390916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f166c0c9-66bf-44f7-b73a-c0b67ae57393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:27.399712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 955/1901] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:27.404364 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932fb69-fc49-4e3f-a290-5a17df0217c8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:27.406460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d932fb69-fc49-4e3f-a290-5a17df0217c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:27.469340 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 947/1902] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:27.476637 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166c0c9-66bf-44f7-b73a-c0b67ae57393 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:27.482743 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f166c0c9-66bf-44f7-b73a-c0b67ae57393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:27.487781 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932fb69-fc49-4e3f-a290-5a17df0217c8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:27.492733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3d00e-73b5-4d1c-8eb3-56ae2db21ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:27.493460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3d00e-73b5-4d1c-8eb3-56ae2db21ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:27.493803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3d00e-73b5-4d1c-8eb3-56ae2db21ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:27.494261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3d00e-73b5-4d1c-8eb3-56ae2db21ab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:27.494967 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee3d00e-73b5-4d1c-8eb3-56ae2db21ab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:27.496259 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166c0c9-66bf-44f7-b73a-c0b67ae57393 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:27.496927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee3d00e-73b5-4d1c-8eb3-56ae2db21ab5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:27.502611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46f7d7-e391-441a-9abd-a028e8a3d1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:27.503811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46f7d7-e391-441a-9abd-a028e8a3d1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:27.504294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46f7d7-e391-441a-9abd-a028e8a3d1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:27.505098 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46f7d7-e391-441a-9abd-a028e8a3d1e3 admin admin] PATH_INFO: `/v3/users/b18f559b26f04bc3afdce16d7a10ac36` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:27.506459 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46f7d7-e391-441a-9abd-a028e8a3d1e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b18f559b26f04bc3afdce16d7a10ac36)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:27.508406 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46f7d7-e391-441a-9abd-a028e8a3d1e3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:27.617531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 956/1903] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:27 2022] POST /identity/v3/projects => generated 379 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:27.617531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3d00e-73b5-4d1c-8eb3-56ae2db21ab5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:27.617531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dee3d00e-73b5-4d1c-8eb3-56ae2db21ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:27.673693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3d00e-73b5-4d1c-8eb3-56ae2db21ab5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:27.677420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e689e-178d-47f6-a688-7dabf372ac6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:27.678107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e689e-178d-47f6-a688-7dabf372ac6b service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:27.678401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e689e-178d-47f6-a688-7dabf372ac6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:27.678687 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e689e-178d-47f6-a688-7dabf372ac6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:27.745529 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e689e-178d-47f6-a688-7dabf372ac6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:27.746960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e689e-178d-47f6-a688-7dabf372ac6b service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:27.757548 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 957/1904] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:27.758387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e689e-178d-47f6-a688-7dabf372ac6b service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:27.769432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e689e-178d-47f6-a688-7dabf372ac6b service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:27.786356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2576316-b349-419b-9969-1ab0250b9c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:27.786356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2576316-b349-419b-9969-1ab0250b9c25 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:27.786356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2576316-b349-419b-9969-1ab0250b9c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:27.786356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2576316-b349-419b-9969-1ab0250b9c25 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:27.786774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2576316-b349-419b-9969-1ab0250b9c25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:27.786774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2576316-b349-419b-9969-1ab0250b9c25 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:27.788492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5e46f7d7-e391-441a-9abd-a028e8a3d1e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b18f559b26f04bc3afdce16d7a10ac36'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:27.789304 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5e46f7d7-e391-441a-9abd-a028e8a3d1e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b18f559b26f04bc3afdce16d7a10ac36'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:27.790229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5e46f7d7-e391-441a-9abd-a028e8a3d1e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b18f559b26f04bc3afdce16d7a10ac36'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:27.802347 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5e46f7d7-e391-441a-9abd-a028e8a3d1e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b18f559b26f04bc3afdce16d7a10ac36'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:27.819500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5e46f7d7-e391-441a-9abd-a028e8a3d1e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b18f559b26f04bc3afdce16d7a10ac36'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:27.929220 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 958/1905] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:27 2022] POST /identity/v3/users => generated 296 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:27.931891 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2576316-b349-419b-9969-1ab0250b9c25 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:27.933711 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b2576316-b349-419b-9969-1ab0250b9c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:27.946622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2576316-b349-419b-9969-1ab0250b9c25 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:27.951489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b6d7d-a412-4051-b85f-ef35df19958f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:27.952778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b6d7d-a412-4051-b85f-ef35df19958f admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:27.953170 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b6d7d-a412-4051-b85f-ef35df19958f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:27.953541 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b6d7d-a412-4051-b85f-ef35df19958f admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:27.954349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b6d7d-a412-4051-b85f-ef35df19958f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:27.956031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b6d7d-a412-4051-b85f-ef35df19958f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:27.958686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 948/1906] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:27 2022] DELETE /identity/v3/users/b18f559b26f04bc3afdce16d7a10ac36 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:27.972423 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 959/1907] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:27.972665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bdd25-bcd6-48ac-9b4e-21de33dfb874 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:27.972665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bdd25-bcd6-48ac-9b4e-21de33dfb874 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:27.972843 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bdd25-bcd6-48ac-9b4e-21de33dfb874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:27.973836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b6d7d-a412-4051-b85f-ef35df19958f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:27.985239 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b6d7d-a412-4051-b85f-ef35df19958f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:27.999319 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-186bdd25-bcd6-48ac-9b4e-21de33dfb874 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:28.000649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-186bdd25-bcd6-48ac-9b4e-21de33dfb874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:28.057289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fe527-6be2-4692-b09c-600baf5463ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:28.058320 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fe527-6be2-4692-b09c-600baf5463ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:28.058687 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fe527-6be2-4692-b09c-600baf5463ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:28.059095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fe527-6be2-4692-b09c-600baf5463ea admin admin] PATH_INFO: `/v3/projects/156e936402d04354a66c9ce917949001/users/a6f8996eab144e52906943c68114ed09/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:28.065236 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:28.074989 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 949/1908] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:28.080044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2fe527-6be2-4692-b09c-600baf5463ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=156e936402d04354a66c9ce917949001, user_id=a6f8996eab144e52906943c68114ed09, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:28.085449 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2fe527-6be2-4692-b09c-600baf5463ea admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:28.090397 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bdd25-bcd6-48ac-9b4e-21de33dfb874 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:28.091985 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-186bdd25-bcd6-48ac-9b4e-21de33dfb874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:28.172461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bdd25-bcd6-48ac-9b4e-21de33dfb874 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:28.176449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553b23c-820d-44ed-82c0-1e41db70fee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:28.177170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553b23c-820d-44ed-82c0-1e41db70fee9 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:28.177460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553b23c-820d-44ed-82c0-1e41db70fee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:28.177928 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553b23c-820d-44ed-82c0-1e41db70fee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:28.187124 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:28.191704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553b23c-820d-44ed-82c0-1e41db70fee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:28.193280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553b23c-820d-44ed-82c0-1e41db70fee9 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:28.204544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 960/1909] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:27 2022] PUT /identity/v3/projects/156e936402d04354a66c9ce917949001/users/a6f8996eab144e52906943c68114ed09/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:28.204544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fe527-6be2-4692-b09c-600baf5463ea admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:28.205028 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 950/1910] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:28.301990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fe527-6be2-4692-b09c-600baf5463ea admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:28.307981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fe78f-c38b-48b2-ad4c-abef85848423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:28.308740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fe78f-c38b-48b2-ad4c-abef85848423 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:28.308977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fe78f-c38b-48b2-ad4c-abef85848423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:28.309304 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fe78f-c38b-48b2-ad4c-abef85848423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:28.310437 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967fe78f-c38b-48b2-ad4c-abef85848423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:28.312730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967fe78f-c38b-48b2-ad4c-abef85848423 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:28.327651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 961/1911] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:28.328315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553b23c-820d-44ed-82c0-1e41db70fee9 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:28.338745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553b23c-820d-44ed-82c0-1e41db70fee9 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:28.360209 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4da44-343d-4688-9a59-0d3ef1ac9956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:28.362260 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4da44-343d-4688-9a59-0d3ef1ac9956 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:28.362955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4da44-343d-4688-9a59-0d3ef1ac9956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:28.363454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4da44-343d-4688-9a59-0d3ef1ac9956 admin admin] PATH_INFO: `/v3/projects/156e936402d04354a66c9ce917949001/users/a6f8996eab144e52906943c68114ed09/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:28.367310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4da44-343d-4688-9a59-0d3ef1ac9956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=156e936402d04354a66c9ce917949001, user_id=a6f8996eab144e52906943c68114ed09, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:28.373523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4da44-343d-4688-9a59-0d3ef1ac9956 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:28.388934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 951/1912] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:28 2022] PUT /identity/v3/projects/156e936402d04354a66c9ce917949001/users/a6f8996eab144e52906943c68114ed09/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:28.400736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81719149-6abc-4887-b004-a6c3e225f511 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:28.401109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81719149-6abc-4887-b004-a6c3e225f511 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:28.401360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81719149-6abc-4887-b004-a6c3e225f511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:28.456341 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-81719149-6abc-4887-b004-a6c3e225f511 None None] MFA Rules not processed for user `a6f8996eab144e52906943c68114ed09`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:28.457906 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-81719149-6abc-4887-b004-a6c3e225f511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:28.537937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 962/1913] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:28.549864 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4da44-343d-4688-9a59-0d3ef1ac9956 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:28.551533 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bfa4da44-343d-4688-9a59-0d3ef1ac9956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:28.597452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81719149-6abc-4887-b004-a6c3e225f511 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:28.597452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-81719149-6abc-4887-b004-a6c3e225f511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:28.632724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4da44-343d-4688-9a59-0d3ef1ac9956 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:28.642957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e35b49-c194-410c-a97f-679e76f1e815 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:28.643609 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e35b49-c194-410c-a97f-679e76f1e815 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:28.643903 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e35b49-c194-410c-a97f-679e76f1e815 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:28.644185 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e35b49-c194-410c-a97f-679e76f1e815 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:28.669174 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e35b49-c194-410c-a97f-679e76f1e815 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:28.671647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e35b49-c194-410c-a97f-679e76f1e815 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:28.687399 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 952/1914] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:28.724376 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81719149-6abc-4887-b004-a6c3e225f511 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:28.730206 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fd1a5-899e-4700-85aa-d96d151bf092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:28.731114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fd1a5-899e-4700-85aa-d96d151bf092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:28.731382 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fd1a5-899e-4700-85aa-d96d151bf092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:28.731698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fd1a5-899e-4700-85aa-d96d151bf092 admin admin] PATH_INFO: `/v3/projects/9b8dad9fc1d2487ab233ffbf7b8b993f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:28.732971 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fd1a5-899e-4700-85aa-d96d151bf092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b8dad9fc1d2487ab233ffbf7b8b993f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:28.735008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fd1a5-899e-4700-85aa-d96d151bf092 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:28.809789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e35b49-c194-410c-a97f-679e76f1e815 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:28.812438 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c7e35b49-c194-410c-a97f-679e76f1e815 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:28.823713 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e35b49-c194-410c-a97f-679e76f1e815 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:28.828121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a6abc-f37f-4fed-b32c-75c432e97ef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:28.828925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a6abc-f37f-4fed-b32c-75c432e97ef6 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:28.829252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a6abc-f37f-4fed-b32c-75c432e97ef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:28.829535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a6abc-f37f-4fed-b32c-75c432e97ef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:28.831334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-336a6abc-f37f-4fed-b32c-75c432e97ef6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:28.890832 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-187fd1a5-899e-4700-85aa-d96d151bf092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b8dad9fc1d2487ab233ffbf7b8b993f', 'request_id': 'req-187fd1a5-899e-4700-85aa-d96d151bf092'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:28.898806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-187fd1a5-899e-4700-85aa-d96d151bf092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b8dad9fc1d2487ab233ffbf7b8b993f', 'request_id': 'req-187fd1a5-899e-4700-85aa-d96d151bf092'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:28.907986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a6abc-f37f-4fed-b32c-75c432e97ef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:28.912859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a6abc-f37f-4fed-b32c-75c432e97ef6 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:28.929204 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 953/1915] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:28.971911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-187fd1a5-899e-4700-85aa-d96d151bf092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b8dad9fc1d2487ab233ffbf7b8b993f', 'request_id': 'req-187fd1a5-899e-4700-85aa-d96d151bf092'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:28.986291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-187fd1a5-899e-4700-85aa-d96d151bf092 admin admin] The token cache is being invalidate because project 9b8dad9fc1d2487ab233ffbf7b8b993f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:28.989269 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-187fd1a5-899e-4700-85aa-d96d151bf092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:28.990742 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-187fd1a5-899e-4700-85aa-d96d151bf092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:28.992415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 963/1916] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:28 2022] DELETE /identity/v3/projects/9b8dad9fc1d2487ab233ffbf7b8b993f => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:28.996720 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a6abc-f37f-4fed-b32c-75c432e97ef6 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:29.008039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a6abc-f37f-4fed-b32c-75c432e97ef6 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:29.075067 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8284e-ff50-4448-b524-b023794e378d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:29.076025 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8284e-ff50-4448-b524-b023794e378d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:29.076359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8284e-ff50-4448-b524-b023794e378d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:29.076703 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8284e-ff50-4448-b524-b023794e378d admin admin] PATH_INFO: `/v3/users/94bedd383da2420db4ca8b0392d1a1c6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:29.078352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8284e-ff50-4448-b524-b023794e378d admin admin] RBAC: Authorizing `identity:delete_user(user_id=94bedd383da2420db4ca8b0392d1a1c6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:29.080780 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8284e-ff50-4448-b524-b023794e378d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:29.098145 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fd1a5-899e-4700-85aa-d96d151bf092 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:29.100644 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-187fd1a5-899e-4700-85aa-d96d151bf092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:29.136211 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fd1a5-899e-4700-85aa-d96d151bf092 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:29.136982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688461af-15fa-4a18-aad8-245be195dbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:29.138840 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688461af-15fa-4a18-aad8-245be195dbce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:29.139325 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688461af-15fa-4a18-aad8-245be195dbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:29.148864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688461af-15fa-4a18-aad8-245be195dbce admin admin] PATH_INFO: `/v3/projects/255c3a1ae37d483f8c21783f7922dfb7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:29.150651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688461af-15fa-4a18-aad8-245be195dbce admin admin] RBAC: Authorizing `identity:delete_project(project_id=255c3a1ae37d483f8c21783f7922dfb7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:29.153154 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688461af-15fa-4a18-aad8-245be195dbce admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:29.330661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-688461af-15fa-4a18-aad8-245be195dbce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '255c3a1ae37d483f8c21783f7922dfb7', 'request_id': 'req-688461af-15fa-4a18-aad8-245be195dbce'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:29.331195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-99f8284e-ff50-4448-b524-b023794e378d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94bedd383da2420db4ca8b0392d1a1c6'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:29.332225 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-99f8284e-ff50-4448-b524-b023794e378d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94bedd383da2420db4ca8b0392d1a1c6'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:29.332939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-99f8284e-ff50-4448-b524-b023794e378d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94bedd383da2420db4ca8b0392d1a1c6'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:29.339639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-688461af-15fa-4a18-aad8-245be195dbce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '255c3a1ae37d483f8c21783f7922dfb7', 'request_id': 'req-688461af-15fa-4a18-aad8-245be195dbce'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:29.340817 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-99f8284e-ff50-4448-b524-b023794e378d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94bedd383da2420db4ca8b0392d1a1c6'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:29.355511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-99f8284e-ff50-4448-b524-b023794e378d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94bedd383da2420db4ca8b0392d1a1c6'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:29.453409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-688461af-15fa-4a18-aad8-245be195dbce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '255c3a1ae37d483f8c21783f7922dfb7', 'request_id': 'req-688461af-15fa-4a18-aad8-245be195dbce'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:29.476448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-688461af-15fa-4a18-aad8-245be195dbce admin admin] The token cache is being invalidate because project 255c3a1ae37d483f8c21783f7922dfb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:29.476819 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-688461af-15fa-4a18-aad8-245be195dbce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:29.477664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-688461af-15fa-4a18-aad8-245be195dbce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:29.483628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 964/1917] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:29 2022] DELETE /identity/v3/projects/255c3a1ae37d483f8c21783f7922dfb7 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:29.484590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688461af-15fa-4a18-aad8-245be195dbce admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:29.486440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-688461af-15fa-4a18-aad8-245be195dbce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:29.528449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 954/1918] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:28 2022] DELETE /identity/v3/users/94bedd383da2420db4ca8b0392d1a1c6 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:29.563354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688461af-15fa-4a18-aad8-245be195dbce admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:29.633633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db17b73-d2be-4dd6-9203-b7e56b6d3c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:29.634655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db17b73-d2be-4dd6-9203-b7e56b6d3c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:29.634947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db17b73-d2be-4dd6-9203-b7e56b6d3c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:29.635229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db17b73-d2be-4dd6-9203-b7e56b6d3c80 admin admin] PATH_INFO: `/v3/users/b1b32d4908c8419ea80a2e92d91b92be` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:29.636609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db17b73-d2be-4dd6-9203-b7e56b6d3c80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1b32d4908c8419ea80a2e92d91b92be)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:29.638811 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db17b73-d2be-4dd6-9203-b7e56b6d3c80 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:29.830700 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:29.939249 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6db17b73-d2be-4dd6-9203-b7e56b6d3c80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1b32d4908c8419ea80a2e92d91b92be'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:29.964773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6db17b73-d2be-4dd6-9203-b7e56b6d3c80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1b32d4908c8419ea80a2e92d91b92be'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.061868 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8284e-ff50-4448-b524-b023794e378d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:30.064339 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:30.064456 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6db17b73-d2be-4dd6-9203-b7e56b6d3c80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1b32d4908c8419ea80a2e92d91b92be'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.067954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6db17b73-d2be-4dd6-9203-b7e56b6d3c80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1b32d4908c8419ea80a2e92d91b92be'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.068584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-99f8284e-ff50-4448-b524-b023794e378d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:30.080036 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6db17b73-d2be-4dd6-9203-b7e56b6d3c80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1b32d4908c8419ea80a2e92d91b92be'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.082486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8284e-ff50-4448-b524-b023794e378d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:30.096634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 965/1919] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:29 2022] DELETE /identity/v3/users/b1b32d4908c8419ea80a2e92d91b92be => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:30.109269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a9cba-7b80-4d4b-8164-44cb1663621a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:30.109269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a9cba-7b80-4d4b-8164-44cb1663621a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:30.109269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a9cba-7b80-4d4b-8164-44cb1663621a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:30.109269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a9cba-7b80-4d4b-8164-44cb1663621a admin admin] PATH_INFO: `/v3/projects/ff880b1e4d5c438d9ed81b8f10972aeb` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:30.110054 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a9cba-7b80-4d4b-8164-44cb1663621a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff880b1e4d5c438d9ed81b8f10972aeb)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:30.111306 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a9cba-7b80-4d4b-8164-44cb1663621a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:30.270720 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0f5a9cba-7b80-4d4b-8164-44cb1663621a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff880b1e4d5c438d9ed81b8f10972aeb', 'request_id': 'req-0f5a9cba-7b80-4d4b-8164-44cb1663621a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.276977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0f5a9cba-7b80-4d4b-8164-44cb1663621a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff880b1e4d5c438d9ed81b8f10972aeb', 'request_id': 'req-0f5a9cba-7b80-4d4b-8164-44cb1663621a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.283442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0f5a9cba-7b80-4d4b-8164-44cb1663621a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff880b1e4d5c438d9ed81b8f10972aeb', 'request_id': 'req-0f5a9cba-7b80-4d4b-8164-44cb1663621a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.450108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0f5a9cba-7b80-4d4b-8164-44cb1663621a admin admin] The token cache is being invalidate because project ff880b1e4d5c438d9ed81b8f10972aeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:30.453177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0f5a9cba-7b80-4d4b-8164-44cb1663621a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.453912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0f5a9cba-7b80-4d4b-8164-44cb1663621a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.455515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 955/1920] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:30 2022] DELETE /identity/v3/projects/ff880b1e4d5c438d9ed81b8f10972aeb => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:30.459584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db17b73-d2be-4dd6-9203-b7e56b6d3c80 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:30.461765 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6db17b73-d2be-4dd6-9203-b7e56b6d3c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:30.533619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db17b73-d2be-4dd6-9203-b7e56b6d3c80 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:30.593148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951a644-39f7-491b-8a67-e53ef6397553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:30.594083 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951a644-39f7-491b-8a67-e53ef6397553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:30.594312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951a644-39f7-491b-8a67-e53ef6397553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:30.594594 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951a644-39f7-491b-8a67-e53ef6397553 admin admin] PATH_INFO: `/v3/users/7d0560b8f8ab4a338fb850ab833cee8b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:30.595818 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951a644-39f7-491b-8a67-e53ef6397553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d0560b8f8ab4a338fb850ab833cee8b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:30.597786 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951a644-39f7-491b-8a67-e53ef6397553 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:30.609417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a9cba-7b80-4d4b-8164-44cb1663621a admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:30.611649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0f5a9cba-7b80-4d4b-8164-44cb1663621a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:30.627491 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a9cba-7b80-4d4b-8164-44cb1663621a admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:30.632306 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d960e-61bf-4d1a-948d-46e8d8c426d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:30.633307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d960e-61bf-4d1a-948d-46e8d8c426d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:30.633667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d960e-61bf-4d1a-948d-46e8d8c426d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:30.634027 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d960e-61bf-4d1a-948d-46e8d8c426d8 admin admin] PATH_INFO: `/v3/projects/e487696322744408baa9bec8fdb0288a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:30.635317 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d960e-61bf-4d1a-948d-46e8d8c426d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e487696322744408baa9bec8fdb0288a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:30.637549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d960e-61bf-4d1a-948d-46e8d8c426d8 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:30.848468 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3951a644-39f7-491b-8a67-e53ef6397553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d0560b8f8ab4a338fb850ab833cee8b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.848765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-080d960e-61bf-4d1a-948d-46e8d8c426d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e487696322744408baa9bec8fdb0288a', 'request_id': 'req-080d960e-61bf-4d1a-948d-46e8d8c426d8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.854682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-080d960e-61bf-4d1a-948d-46e8d8c426d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e487696322744408baa9bec8fdb0288a', 'request_id': 'req-080d960e-61bf-4d1a-948d-46e8d8c426d8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.861081 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-080d960e-61bf-4d1a-948d-46e8d8c426d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e487696322744408baa9bec8fdb0288a', 'request_id': 'req-080d960e-61bf-4d1a-948d-46e8d8c426d8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.861741 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3951a644-39f7-491b-8a67-e53ef6397553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d0560b8f8ab4a338fb850ab833cee8b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.959914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-080d960e-61bf-4d1a-948d-46e8d8c426d8 admin admin] The token cache is being invalidate because project e487696322744408baa9bec8fdb0288a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:30.960210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-080d960e-61bf-4d1a-948d-46e8d8c426d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.960952 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-080d960e-61bf-4d1a-948d-46e8d8c426d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:30.962710 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 956/1921] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:30 2022] DELETE /identity/v3/projects/e487696322744408baa9bec8fdb0288a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:30.967533 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ffa77-892f-4c5f-a5c6-4c9f2523cadf None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:30.967982 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ffa77-892f-4c5f-a5c6-4c9f2523cadf None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:30.968282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ffa77-892f-4c5f-a5c6-4c9f2523cadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:30.985995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c63ffa77-892f-4c5f-a5c6-4c9f2523cadf None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:30.987198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c63ffa77-892f-4c5f-a5c6-4c9f2523cadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:31.006817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3951a644-39f7-491b-8a67-e53ef6397553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d0560b8f8ab4a338fb850ab833cee8b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:31.007500 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3951a644-39f7-491b-8a67-e53ef6397553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d0560b8f8ab4a338fb850ab833cee8b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:31.015890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3951a644-39f7-491b-8a67-e53ef6397553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d0560b8f8ab4a338fb850ab833cee8b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:31.023380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 966/1922] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:30 2022] DELETE /identity/v3/users/7d0560b8f8ab4a338fb850ab833cee8b => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:31.050867 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 957/1923] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:31.064812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373a8d8-ee4d-40df-a427-020a5fd82527 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:31.073043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373a8d8-ee4d-40df-a427-020a5fd82527 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:31.073043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373a8d8-ee4d-40df-a427-020a5fd82527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:31.084412 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8373a8d8-ee4d-40df-a427-020a5fd82527 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:31.085779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8373a8d8-ee4d-40df-a427-020a5fd82527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:31.146731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 967/1924] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:31.151289 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ffa77-892f-4c5f-a5c6-4c9f2523cadf None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:31.161116 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ffa77-892f-4c5f-a5c6-4c9f2523cadf None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:31.165661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf867a2-f5d1-4e7e-9545-08594aa66849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:31.166388 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf867a2-f5d1-4e7e-9545-08594aa66849 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:31.166674 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf867a2-f5d1-4e7e-9545-08594aa66849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:31.167016 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf867a2-f5d1-4e7e-9545-08594aa66849 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:31.167750 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf867a2-f5d1-4e7e-9545-08594aa66849 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:31.168345 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf867a2-f5d1-4e7e-9545-08594aa66849 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:31.170959 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf867a2-f5d1-4e7e-9545-08594aa66849 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:31.182506 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 958/1925] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:42:31.187470 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373a8d8-ee4d-40df-a427-020a5fd82527 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:31.196671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373a8d8-ee4d-40df-a427-020a5fd82527 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:31.201857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff4292-89c8-4cb2-86f5-3a83b2aef866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:31.202656 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff4292-89c8-4cb2-86f5-3a83b2aef866 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:31.202955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff4292-89c8-4cb2-86f5-3a83b2aef866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:31.203240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff4292-89c8-4cb2-86f5-3a83b2aef866 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:31.203946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff4292-89c8-4cb2-86f5-3a83b2aef866 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:31.206008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff4292-89c8-4cb2-86f5-3a83b2aef866 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:31.300836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 968/1926] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:31 2022] POST /identity/v3/projects => generated 389 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:31.305153 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf867a2-f5d1-4e7e-9545-08594aa66849 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:31.316303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf867a2-f5d1-4e7e-9545-08594aa66849 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:31.372344 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a639ce-95b2-4e73-81e7-c99a89688c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:31.372948 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a639ce-95b2-4e73-81e7-c99a89688c9b admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:31.373284 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a639ce-95b2-4e73-81e7-c99a89688c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:31.374699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a639ce-95b2-4e73-81e7-c99a89688c9b admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:31.375408 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a639ce-95b2-4e73-81e7-c99a89688c9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:31.377368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a639ce-95b2-4e73-81e7-c99a89688c9b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:31.421436 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff4292-89c8-4cb2-86f5-3a83b2aef866 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:31.423258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-19ff4292-89c8-4cb2-86f5-3a83b2aef866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:31.438854 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff4292-89c8-4cb2-86f5-3a83b2aef866 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:31.443923 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5fe0e-78ac-4cf9-abc9-30939a726d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:31.444976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5fe0e-78ac-4cf9-abc9-30939a726d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:31.445300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5fe0e-78ac-4cf9-abc9-30939a726d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:31.445871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5fe0e-78ac-4cf9-abc9-30939a726d31 admin admin] PATH_INFO: `/v3/projects/b846b65ddcd24d4eb2f4db2a63e509b0` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:31.450135 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:31.451575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5fe0e-78ac-4cf9-abc9-30939a726d31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b846b65ddcd24d4eb2f4db2a63e509b0)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:31.459660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5fe0e-78ac-4cf9-abc9-30939a726d31 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:31.485258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 959/1927] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:31 2022] POST /identity/v3/users => generated 295 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:31.501230 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a639ce-95b2-4e73-81e7-c99a89688c9b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:31.511354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a639ce-95b2-4e73-81e7-c99a89688c9b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:31.516535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b38157-170a-4bac-a759-0e9a837ed70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:31.517274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b38157-170a-4bac-a759-0e9a837ed70f admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:31.517559 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b38157-170a-4bac-a759-0e9a837ed70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:31.517847 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b38157-170a-4bac-a759-0e9a837ed70f admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:31.518588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b38157-170a-4bac-a759-0e9a837ed70f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:31.520445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b38157-170a-4bac-a759-0e9a837ed70f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:31.531747 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 960/1928] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:31.535751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b38157-170a-4bac-a759-0e9a837ed70f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:31.546581 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b38157-170a-4bac-a759-0e9a837ed70f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:31.551214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa341afb-b1a2-4604-a4c7-bf26991376b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:31.551873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa341afb-b1a2-4604-a4c7-bf26991376b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:31.552172 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa341afb-b1a2-4604-a4c7-bf26991376b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:31.552461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa341afb-b1a2-4604-a4c7-bf26991376b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:31.553230 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa341afb-b1a2-4604-a4c7-bf26991376b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:31.554837 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa341afb-b1a2-4604-a4c7-bf26991376b2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:31.568412 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 961/1929] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:31.569343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa341afb-b1a2-4604-a4c7-bf26991376b2 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:31.580098 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa341afb-b1a2-4604-a4c7-bf26991376b2 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:31.594733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-47a5fe0e-78ac-4cf9-abc9-30939a726d31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b846b65ddcd24d4eb2f4db2a63e509b0', 'request_id': 'req-47a5fe0e-78ac-4cf9-abc9-30939a726d31'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:31.603883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-47a5fe0e-78ac-4cf9-abc9-30939a726d31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b846b65ddcd24d4eb2f4db2a63e509b0', 'request_id': 'req-47a5fe0e-78ac-4cf9-abc9-30939a726d31'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:31.641250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97117e-b937-4f67-9251-2c0011aaf62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:31.643435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97117e-b937-4f67-9251-2c0011aaf62b admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:31.643435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97117e-b937-4f67-9251-2c0011aaf62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:31.643435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97117e-b937-4f67-9251-2c0011aaf62b admin admin] PATH_INFO: `/v3/projects/223f8dc992134a258a3bfd5ba8dd1ee9/users/ff7846829c90496d99b942fadc1ddfc4/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:31.657552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97117e-b937-4f67-9251-2c0011aaf62b admin admin] RBAC: Authorizing `identity:create_grant(project_id=223f8dc992134a258a3bfd5ba8dd1ee9, user_id=ff7846829c90496d99b942fadc1ddfc4, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:31.660288 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97117e-b937-4f67-9251-2c0011aaf62b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:31.663942 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:31.686307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-47a5fe0e-78ac-4cf9-abc9-30939a726d31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b846b65ddcd24d4eb2f4db2a63e509b0', 'request_id': 'req-47a5fe0e-78ac-4cf9-abc9-30939a726d31'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:31.697711 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-47a5fe0e-78ac-4cf9-abc9-30939a726d31 admin admin] The token cache is being invalidate because project b846b65ddcd24d4eb2f4db2a63e509b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:31.698005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-47a5fe0e-78ac-4cf9-abc9-30939a726d31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:31.698685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-47a5fe0e-78ac-4cf9-abc9-30939a726d31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:31.703747 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 969/1930] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:31 2022] DELETE /identity/v3/projects/b846b65ddcd24d4eb2f4db2a63e509b0 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:31.706168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e459c25-1541-499c-a2da-a5e85d668ad6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:31.706469 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e459c25-1541-499c-a2da-a5e85d668ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:31.706753 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e459c25-1541-499c-a2da-a5e85d668ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:31.724783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0e459c25-1541-499c-a2da-a5e85d668ad6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:31.726018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0e459c25-1541-499c-a2da-a5e85d668ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:31.734640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 970/1931] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:31.743650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ed65d-cf8f-428c-b657-cff2aebc16d6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:31.743947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ed65d-cf8f-428c-b657-cff2aebc16d6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:31.746010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 962/1932] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:31 2022] PUT /identity/v3/projects/223f8dc992134a258a3bfd5ba8dd1ee9/users/ff7846829c90496d99b942fadc1ddfc4/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:31.746126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ed65d-cf8f-428c-b657-cff2aebc16d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:31.755293 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eec76f-f963-41b6-b110-5234ea1e59e5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:31.755597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eec76f-f963-41b6-b110-5234ea1e59e5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:31.755883 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eec76f-f963-41b6-b110-5234ea1e59e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:31.762210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ed9ed65d-cf8f-428c-b657-cff2aebc16d6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:31.763415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ed9ed65d-cf8f-428c-b657-cff2aebc16d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:31.784175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-90eec76f-f963-41b6-b110-5234ea1e59e5 None None] MFA Rules not processed for user `ff7846829c90496d99b942fadc1ddfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:31.785549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-90eec76f-f963-41b6-b110-5234ea1e59e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:31.823753 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 971/1933] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:31.828623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ed65d-cf8f-428c-b657-cff2aebc16d6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:31.837356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ed65d-cf8f-428c-b657-cff2aebc16d6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:31.842052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413842a3-baaa-4f20-9cce-a0ddc4b0dbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:31.843262 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413842a3-baaa-4f20-9cce-a0ddc4b0dbde admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:31.843544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413842a3-baaa-4f20-9cce-a0ddc4b0dbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:31.843818 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413842a3-baaa-4f20-9cce-a0ddc4b0dbde admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:31.844540 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413842a3-baaa-4f20-9cce-a0ddc4b0dbde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:31.845001 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413842a3-baaa-4f20-9cce-a0ddc4b0dbde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:31.846586 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413842a3-baaa-4f20-9cce-a0ddc4b0dbde admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:31.854568 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 963/1934] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:31.859260 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eec76f-f963-41b6-b110-5234ea1e59e5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:31.859891 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 972/1935] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:42:31.861833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-90eec76f-f963-41b6-b110-5234ea1e59e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:31.863983 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413842a3-baaa-4f20-9cce-a0ddc4b0dbde admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:31.873475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413842a3-baaa-4f20-9cce-a0ddc4b0dbde admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:31.875262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eec76f-f963-41b6-b110-5234ea1e59e5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:31.879913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9084ec9-15dc-47e2-8082-e92c9f4ea742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:31.880065 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313df01-4600-4275-a0e8-66f5c4de222b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:31.880731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313df01-4600-4275-a0e8-66f5c4de222b admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:31.880869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9084ec9-15dc-47e2-8082-e92c9f4ea742 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:31.881012 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313df01-4600-4275-a0e8-66f5c4de222b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:31.886118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9084ec9-15dc-47e2-8082-e92c9f4ea742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:31.886248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313df01-4600-4275-a0e8-66f5c4de222b admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:31.886355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9084ec9-15dc-47e2-8082-e92c9f4ea742 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:31.886433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313df01-4600-4275-a0e8-66f5c4de222b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:31.886505 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9084ec9-15dc-47e2-8082-e92c9f4ea742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:31.886574 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313df01-4600-4275-a0e8-66f5c4de222b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:31.886653 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9084ec9-15dc-47e2-8082-e92c9f4ea742 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:32.024866 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 964/1936] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:31 2022] POST /identity/v3/projects => generated 389 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:32.028858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313df01-4600-4275-a0e8-66f5c4de222b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:32.055735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 973/1937] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:31 2022] POST /identity/v3/projects => generated 365 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:32.063875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9084ec9-15dc-47e2-8082-e92c9f4ea742 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:32.094226 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313df01-4600-4275-a0e8-66f5c4de222b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:32.136305 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9084ec9-15dc-47e2-8082-e92c9f4ea742 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:32.143144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fed86-b9a8-4ab9-88cd-cc483988779f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:32.144051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fed86-b9a8-4ab9-88cd-cc483988779f admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:32.144437 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fed86-b9a8-4ab9-88cd-cc483988779f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:32.144770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fed86-b9a8-4ab9-88cd-cc483988779f admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:32.145602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fed86-b9a8-4ab9-88cd-cc483988779f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:32.149838 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fed86-b9a8-4ab9-88cd-cc483988779f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:32.155983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc4c0f-ac02-4d0d-9de2-2bb2456e3306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:32.156676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc4c0f-ac02-4d0d-9de2-2bb2456e3306 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:32.157281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc4c0f-ac02-4d0d-9de2-2bb2456e3306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:32.157281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc4c0f-ac02-4d0d-9de2-2bb2456e3306 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:32.158953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc4c0f-ac02-4d0d-9de2-2bb2456e3306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:32.160152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc4c0f-ac02-4d0d-9de2-2bb2456e3306 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:32.241741 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 974/1938] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:32 2022] POST /identity/v3/users => generated 283 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:32.245686 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fed86-b9a8-4ab9-88cd-cc483988779f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:32.256077 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fed86-b9a8-4ab9-88cd-cc483988779f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:32.260616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb55c6-66bf-4afd-93a9-a86de226f545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:32.261287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb55c6-66bf-4afd-93a9-a86de226f545 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:32.261572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb55c6-66bf-4afd-93a9-a86de226f545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:32.262019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb55c6-66bf-4afd-93a9-a86de226f545 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:32.262639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb55c6-66bf-4afd-93a9-a86de226f545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:32.264545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb55c6-66bf-4afd-93a9-a86de226f545 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:32.270320 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 965/1939] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:32 2022] POST /identity/v3/users => generated 301 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:32.274385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc4c0f-ac02-4d0d-9de2-2bb2456e3306 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:32.278831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 975/1940] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:32.280316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb55c6-66bf-4afd-93a9-a86de226f545 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:32.284074 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc4c0f-ac02-4d0d-9de2-2bb2456e3306 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:32.288965 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e56f55-1464-4b34-900c-e098d2b002ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:32.289647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e56f55-1464-4b34-900c-e098d2b002ba admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:32.289938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e56f55-1464-4b34-900c-e098d2b002ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:32.290222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e56f55-1464-4b34-900c-e098d2b002ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:32.290313 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb55c6-66bf-4afd-93a9-a86de226f545 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:32.290949 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e56f55-1464-4b34-900c-e098d2b002ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:32.292708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e56f55-1464-4b34-900c-e098d2b002ba admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:32.296565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdfda4-5af5-4c4f-a2e5-9fffe5f0236a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:32.297284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdfda4-5af5-4c4f-a2e5-9fffe5f0236a admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:32.297598 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdfda4-5af5-4c4f-a2e5-9fffe5f0236a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:32.297865 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdfda4-5af5-4c4f-a2e5-9fffe5f0236a admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:32.298616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdfda4-5af5-4c4f-a2e5-9fffe5f0236a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:32.300203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdfda4-5af5-4c4f-a2e5-9fffe5f0236a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:32.303060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 966/1941] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:32.307174 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e56f55-1464-4b34-900c-e098d2b002ba admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:32.315370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 976/1942] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:32.316640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdfda4-5af5-4c4f-a2e5-9fffe5f0236a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:32.317657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e56f55-1464-4b34-900c-e098d2b002ba admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:32.322428 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5ffde-737c-4f91-b5c3-b03cf5ddd654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:32.323307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5ffde-737c-4f91-b5c3-b03cf5ddd654 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:32.323619 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5ffde-737c-4f91-b5c3-b03cf5ddd654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:32.323896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5ffde-737c-4f91-b5c3-b03cf5ddd654 admin admin] PATH_INFO: `/v3/projects/432f37ad628b4aedb323e795b1e3b6ab/users/d9b5b51e1719476a8b7820f0a6093de5/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:32.327663 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdfda4-5af5-4c4f-a2e5-9fffe5f0236a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:32.332474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b2d7e-2108-43a2-9b79-d6d8ea253d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:32.333414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b2d7e-2108-43a2-9b79-d6d8ea253d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:32.339464 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b2d7e-2108-43a2-9b79-d6d8ea253d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:32.339464 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b2d7e-2108-43a2-9b79-d6d8ea253d5f admin admin] PATH_INFO: `/v3/projects/c892b37d74de4f8eae6be3871be771d3/users/7b83a14b8fcc4492b37c4da4db8edeff/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:32.341360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5ffde-737c-4f91-b5c3-b03cf5ddd654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=432f37ad628b4aedb323e795b1e3b6ab, user_id=d9b5b51e1719476a8b7820f0a6093de5, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:32.344472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5ffde-737c-4f91-b5c3-b03cf5ddd654 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:32.348610 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b2d7e-2108-43a2-9b79-d6d8ea253d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c892b37d74de4f8eae6be3871be771d3, user_id=7b83a14b8fcc4492b37c4da4db8edeff, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:32.351318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b2d7e-2108-43a2-9b79-d6d8ea253d5f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:32.356335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:32.435094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 967/1943] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:32 2022] PUT /identity/v3/projects/432f37ad628b4aedb323e795b1e3b6ab/users/d9b5b51e1719476a8b7820f0a6093de5/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:32.439461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5ffde-737c-4f91-b5c3-b03cf5ddd654 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:32.447393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 977/1944] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:32 2022] PUT /identity/v3/projects/c892b37d74de4f8eae6be3871be771d3/users/7b83a14b8fcc4492b37c4da4db8edeff/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:32.459457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7217c7-e194-4417-9b62-9d78e52e1ca1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:32.459745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7217c7-e194-4417-9b62-9d78e52e1ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:32.460021 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7217c7-e194-4417-9b62-9d78e52e1ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:32.488491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9e7217c7-e194-4417-9b62-9d78e52e1ca1 None None] MFA Rules not processed for user `7b83a14b8fcc4492b37c4da4db8edeff`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:32.489759 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9e7217c7-e194-4417-9b62-9d78e52e1ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:32.503401 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5ffde-737c-4f91-b5c3-b03cf5ddd654 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:32.555365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 978/1945] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:32.559820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7217c7-e194-4417-9b62-9d78e52e1ca1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:32.563188 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:32.564920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9c7fe-c324-46fa-9dc8-c27384d8af14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:32.565639 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9c7fe-c324-46fa-9dc8-c27384d8af14 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:32.565932 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9c7fe-c324-46fa-9dc8-c27384d8af14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:32.566215 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9c7fe-c324-46fa-9dc8-c27384d8af14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:32.566955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9c7fe-c324-46fa-9dc8-c27384d8af14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:32.568562 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9c7fe-c324-46fa-9dc8-c27384d8af14 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:32.579101 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 968/1946] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:32.582976 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9c7fe-c324-46fa-9dc8-c27384d8af14 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:32.593200 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9c7fe-c324-46fa-9dc8-c27384d8af14 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:32.597985 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c2ac3-e1b4-4926-9cde-391975e62f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:32.598866 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c2ac3-e1b4-4926-9cde-391975e62f5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:32.599158 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c2ac3-e1b4-4926-9cde-391975e62f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:32.599443 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c2ac3-e1b4-4926-9cde-391975e62f5d admin admin] PATH_INFO: `/v3/projects/432f37ad628b4aedb323e795b1e3b6ab/users/d9b5b51e1719476a8b7820f0a6093de5/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:32.601460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c2ac3-e1b4-4926-9cde-391975e62f5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=432f37ad628b4aedb323e795b1e3b6ab, user_id=d9b5b51e1719476a8b7820f0a6093de5, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:32.604122 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c2ac3-e1b4-4926-9cde-391975e62f5d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:32.614459 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 969/1947] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:32 2022] PUT /identity/v3/projects/432f37ad628b4aedb323e795b1e3b6ab/users/d9b5b51e1719476a8b7820f0a6093de5/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:32.625354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7217c7-e194-4417-9b62-9d78e52e1ca1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:32.625846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a18df-60ea-4afa-9a1b-c99247992cb9 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:32.626095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a18df-60ea-4afa-9a1b-c99247992cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:32.626390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a18df-60ea-4afa-9a1b-c99247992cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:32.658128 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ba9a18df-60ea-4afa-9a1b-c99247992cb9 None None] MFA Rules not processed for user `d9b5b51e1719476a8b7820f0a6093de5`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:32.658936 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ba9a18df-60ea-4afa-9a1b-c99247992cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:32.708734 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703c6a0-3711-4bd0-bbaf-eb4a3839c9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:32.709490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703c6a0-3711-4bd0-bbaf-eb4a3839c9a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:32.709783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703c6a0-3711-4bd0-bbaf-eb4a3839c9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:32.710078 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703c6a0-3711-4bd0-bbaf-eb4a3839c9a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:32.710766 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703c6a0-3711-4bd0-bbaf-eb4a3839c9a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:32.713220 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703c6a0-3711-4bd0-bbaf-eb4a3839c9a5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:32.751997 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 970/1948] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:32.759697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a18df-60ea-4afa-9a1b-c99247992cb9 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:32.761107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ba9a18df-60ea-4afa-9a1b-c99247992cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:32.812570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 979/1949] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:32 2022] POST /identity/v3/projects => generated 367 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:32.819469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a18df-60ea-4afa-9a1b-c99247992cb9 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:32.821551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703c6a0-3711-4bd0-bbaf-eb4a3839c9a5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:32.876650 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742f0e8-1b8e-4009-aec5-27e4db55eea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:32.877478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742f0e8-1b8e-4009-aec5-27e4db55eea4 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:32.877768 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742f0e8-1b8e-4009-aec5-27e4db55eea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:32.878048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742f0e8-1b8e-4009-aec5-27e4db55eea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:32.893279 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703c6a0-3711-4bd0-bbaf-eb4a3839c9a5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:32.898069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d96c6-66a3-4638-9999-76cd900f1a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:32.898748 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d96c6-66a3-4638-9999-76cd900f1a8a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:32.899035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d96c6-66a3-4638-9999-76cd900f1a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:32.899314 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d96c6-66a3-4638-9999-76cd900f1a8a admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:32.900037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d96c6-66a3-4638-9999-76cd900f1a8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:32.901994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d96c6-66a3-4638-9999-76cd900f1a8a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:32.940918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742f0e8-1b8e-4009-aec5-27e4db55eea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:32.942393 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742f0e8-1b8e-4009-aec5-27e4db55eea4 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:32.952406 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 971/1950] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:32.967676 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 980/1951] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:32 2022] POST /identity/v3/users => generated 290 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:32.971619 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742f0e8-1b8e-4009-aec5-27e4db55eea4 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:32.982264 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742f0e8-1b8e-4009-aec5-27e4db55eea4 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:32.985492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:32.989388 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1365c5c-8391-489f-b027-30e322eca055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:32.994673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1365c5c-8391-489f-b027-30e322eca055 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:32.994673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1365c5c-8391-489f-b027-30e322eca055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:32.994673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1365c5c-8391-489f-b027-30e322eca055 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:32.994673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1365c5c-8391-489f-b027-30e322eca055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:32.994673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1365c5c-8391-489f-b027-30e322eca055 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:33.007173 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 972/1952] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:33.008824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d96c6-66a3-4638-9999-76cd900f1a8a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:33.024491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d96c6-66a3-4638-9999-76cd900f1a8a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:33.024491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ab59a-81a4-42b3-9936-04556b2423cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:33.025333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ab59a-81a4-42b3-9936-04556b2423cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:33.025625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ab59a-81a4-42b3-9936-04556b2423cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:33.025933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ab59a-81a4-42b3-9936-04556b2423cd admin admin] PATH_INFO: `/v3/projects/694e8fcffbe743b8a332c47c91cba2cf/users/e206013b6cb04bb2ac8522f44d507393/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:33.041879 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ab59a-81a4-42b3-9936-04556b2423cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=694e8fcffbe743b8a332c47c91cba2cf, user_id=e206013b6cb04bb2ac8522f44d507393, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:33.044820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ab59a-81a4-42b3-9936-04556b2423cd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:33.081858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1365c5c-8391-489f-b027-30e322eca055 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:33.091841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1365c5c-8391-489f-b027-30e322eca055 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:33.097052 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49885f-7c62-4366-8999-15662d6a6d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:33.098018 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49885f-7c62-4366-8999-15662d6a6d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:33.098318 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49885f-7c62-4366-8999-15662d6a6d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:33.098603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49885f-7c62-4366-8999-15662d6a6d85 admin admin] PATH_INFO: `/v3/users/ff7846829c90496d99b942fadc1ddfc4` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:33.099812 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49885f-7c62-4366-8999-15662d6a6d85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff7846829c90496d99b942fadc1ddfc4)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:33.101851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49885f-7c62-4366-8999-15662d6a6d85 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:33.149572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 981/1953] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:33 2022] PUT /identity/v3/projects/694e8fcffbe743b8a332c47c91cba2cf/users/e206013b6cb04bb2ac8522f44d507393/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:33.153445 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ab59a-81a4-42b3-9936-04556b2423cd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:33.168487 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:33.230685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ab59a-81a4-42b3-9936-04556b2423cd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:33.242379 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e205c-8971-4cde-98b2-8c071d7a5ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:33.244207 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e205c-8971-4cde-98b2-8c071d7a5ebb admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:33.244348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e205c-8971-4cde-98b2-8c071d7a5ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:33.244573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e205c-8971-4cde-98b2-8c071d7a5ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:33.245308 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e205c-8971-4cde-98b2-8c071d7a5ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:33.247041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e205c-8971-4cde-98b2-8c071d7a5ebb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:33.258203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 982/1954] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:33.262331 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e205c-8971-4cde-98b2-8c071d7a5ebb admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:33.273158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e205c-8971-4cde-98b2-8c071d7a5ebb admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:33.278856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087975ae-3178-409b-9324-c15ab3a032d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:33.279789 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087975ae-3178-409b-9324-c15ab3a032d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:33.280782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087975ae-3178-409b-9324-c15ab3a032d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:33.281095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087975ae-3178-409b-9324-c15ab3a032d1 admin admin] PATH_INFO: `/v3/projects/694e8fcffbe743b8a332c47c91cba2cf/users/e206013b6cb04bb2ac8522f44d507393/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:33.284305 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087975ae-3178-409b-9324-c15ab3a032d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=694e8fcffbe743b8a332c47c91cba2cf, user_id=e206013b6cb04bb2ac8522f44d507393, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:33.287029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087975ae-3178-409b-9324-c15ab3a032d1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:33.300590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 983/1955] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:33 2022] PUT /identity/v3/projects/694e8fcffbe743b8a332c47c91cba2cf/users/e206013b6cb04bb2ac8522f44d507393/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:33.310537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62899a3b-e38f-4563-b055-992d893faf85 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:33.311983 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62899a3b-e38f-4563-b055-992d893faf85 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:33.311983 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62899a3b-e38f-4563-b055-992d893faf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:33.340515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-62899a3b-e38f-4563-b055-992d893faf85 None None] MFA Rules not processed for user `e206013b6cb04bb2ac8522f44d507393`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:33.341774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-62899a3b-e38f-4563-b055-992d893faf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:33.345621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4c49885f-7c62-4366-8999-15662d6a6d85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff7846829c90496d99b942fadc1ddfc4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:33.346560 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4c49885f-7c62-4366-8999-15662d6a6d85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff7846829c90496d99b942fadc1ddfc4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:33.347666 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4c49885f-7c62-4366-8999-15662d6a6d85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff7846829c90496d99b942fadc1ddfc4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:33.355355 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4c49885f-7c62-4366-8999-15662d6a6d85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff7846829c90496d99b942fadc1ddfc4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:33.370108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4c49885f-7c62-4366-8999-15662d6a6d85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff7846829c90496d99b942fadc1ddfc4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:33.411539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 984/1956] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:33 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:33.427312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62899a3b-e38f-4563-b055-992d893faf85 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:33.428901 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-62899a3b-e38f-4563-b055-992d893faf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:33.477375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 973/1957] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:33 2022] DELETE /identity/v3/users/ff7846829c90496d99b942fadc1ddfc4 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:33.482368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b4ebb-4dcf-4eaa-bb69-84867da05994 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:33.482667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b4ebb-4dcf-4eaa-bb69-84867da05994 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:33.483588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b4ebb-4dcf-4eaa-bb69-84867da05994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:33.503063 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62899a3b-e38f-4563-b055-992d893faf85 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:33.504294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fb4b4ebb-4dcf-4eaa-bb69-84867da05994 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:33.505636 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fb4b4ebb-4dcf-4eaa-bb69-84867da05994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:33.546364 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:33.557460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6529f-5c55-4a5f-9267-295f55d38f1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:33.558348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6529f-5c55-4a5f-9267-295f55d38f1f service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:33.558348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6529f-5c55-4a5f-9267-295f55d38f1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:33.558687 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6529f-5c55-4a5f-9267-295f55d38f1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:33.592845 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 974/1958] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:33.602995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b4ebb-4dcf-4eaa-bb69-84867da05994 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:33.604708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fb4b4ebb-4dcf-4eaa-bb69-84867da05994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:33.632893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6529f-5c55-4a5f-9267-295f55d38f1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:33.634479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6529f-5c55-4a5f-9267-295f55d38f1f service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:33.649209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 985/1959] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:33 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:33.675463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b4ebb-4dcf-4eaa-bb69-84867da05994 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:33.679079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4e7c5-70e6-465d-8d22-3c368f21dd5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:33.679741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4e7c5-70e6-465d-8d22-3c368f21dd5a service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:33.680339 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4e7c5-70e6-465d-8d22-3c368f21dd5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:33.680619 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4e7c5-70e6-465d-8d22-3c368f21dd5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:33.690806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4e7c5-70e6-465d-8d22-3c368f21dd5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:33.692338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4e7c5-70e6-465d-8d22-3c368f21dd5a service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:33.707180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 975/1960] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:33.936721 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6529f-5c55-4a5f-9267-295f55d38f1f service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:33.938342 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a6b6529f-5c55-4a5f-9267-295f55d38f1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:33.949660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6529f-5c55-4a5f-9267-295f55d38f1f service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:33.954894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ddb5f-4902-467e-a4df-feddd1ac04ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:33.955665 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ddb5f-4902-467e-a4df-feddd1ac04ef service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:33.956053 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ddb5f-4902-467e-a4df-feddd1ac04ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:33.956400 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ddb5f-4902-467e-a4df-feddd1ac04ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:33.958557 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2d6ddb5f-4902-467e-a4df-feddd1ac04ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:34.041100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ddb5f-4902-467e-a4df-feddd1ac04ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:34.042937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ddb5f-4902-467e-a4df-feddd1ac04ef service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:34.058282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 986/1961] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:34.278921 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4e7c5-70e6-465d-8d22-3c368f21dd5a service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:34.283039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-00d4e7c5-70e6-465d-8d22-3c368f21dd5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:34.301439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4e7c5-70e6-465d-8d22-3c368f21dd5a service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:34.308247 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80133ba-6df4-49c4-93a8-919b2a5c37f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:34.309319 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80133ba-6df4-49c4-93a8-919b2a5c37f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:34.309816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80133ba-6df4-49c4-93a8-919b2a5c37f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:34.310326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80133ba-6df4-49c4-93a8-919b2a5c37f0 admin admin] PATH_INFO: `/v3/projects/223f8dc992134a258a3bfd5ba8dd1ee9` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:34.312004 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80133ba-6df4-49c4-93a8-919b2a5c37f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=223f8dc992134a258a3bfd5ba8dd1ee9)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:34.322924 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80133ba-6df4-49c4-93a8-919b2a5c37f0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:34.541778 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d80133ba-6df4-49c4-93a8-919b2a5c37f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '223f8dc992134a258a3bfd5ba8dd1ee9', 'request_id': 'req-d80133ba-6df4-49c4-93a8-919b2a5c37f0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:34.547598 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d80133ba-6df4-49c4-93a8-919b2a5c37f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '223f8dc992134a258a3bfd5ba8dd1ee9', 'request_id': 'req-d80133ba-6df4-49c4-93a8-919b2a5c37f0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:34.553785 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d80133ba-6df4-49c4-93a8-919b2a5c37f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '223f8dc992134a258a3bfd5ba8dd1ee9', 'request_id': 'req-d80133ba-6df4-49c4-93a8-919b2a5c37f0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:34.687749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d80133ba-6df4-49c4-93a8-919b2a5c37f0 admin admin] The token cache is being invalidate because project 223f8dc992134a258a3bfd5ba8dd1ee9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:34.688324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d80133ba-6df4-49c4-93a8-919b2a5c37f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:34.688727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d80133ba-6df4-49c4-93a8-919b2a5c37f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:34.690504 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 976/1962] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:34 2022] DELETE /identity/v3/projects/223f8dc992134a258a3bfd5ba8dd1ee9 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:34.699371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ddb5f-4902-467e-a4df-feddd1ac04ef service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:34.708680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ddb5f-4902-467e-a4df-feddd1ac04ef service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:34.776788 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb95e0d-dbe2-4951-83d3-6b33ea172436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:34.777720 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb95e0d-dbe2-4951-83d3-6b33ea172436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:34.778010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb95e0d-dbe2-4951-83d3-6b33ea172436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:34.778309 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb95e0d-dbe2-4951-83d3-6b33ea172436 admin admin] PATH_INFO: `/v3/users/d9b5b51e1719476a8b7820f0a6093de5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:34.779500 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb95e0d-dbe2-4951-83d3-6b33ea172436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9b5b51e1719476a8b7820f0a6093de5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:34.781535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb95e0d-dbe2-4951-83d3-6b33ea172436 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:34.999036 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8cb95e0d-dbe2-4951-83d3-6b33ea172436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9b5b51e1719476a8b7820f0a6093de5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:35.012350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8cb95e0d-dbe2-4951-83d3-6b33ea172436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9b5b51e1719476a8b7820f0a6093de5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:35.099020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8cb95e0d-dbe2-4951-83d3-6b33ea172436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9b5b51e1719476a8b7820f0a6093de5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:35.099725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8cb95e0d-dbe2-4951-83d3-6b33ea172436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9b5b51e1719476a8b7820f0a6093de5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:35.107711 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8cb95e0d-dbe2-4951-83d3-6b33ea172436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9b5b51e1719476a8b7820f0a6093de5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:35.114619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 987/1963] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:34 2022] DELETE /identity/v3/users/d9b5b51e1719476a8b7820f0a6093de5 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:35.207861 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:35.561136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:35.655610 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80133ba-6df4-49c4-93a8-919b2a5c37f0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:35.657512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d80133ba-6df4-49c4-93a8-919b2a5c37f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:35.671186 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80133ba-6df4-49c4-93a8-919b2a5c37f0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:35.729334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cfbc8-f573-4693-a74d-99720f84f5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:35.730211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cfbc8-f573-4693-a74d-99720f84f5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:35.730497 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cfbc8-f573-4693-a74d-99720f84f5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:35.730775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cfbc8-f573-4693-a74d-99720f84f5a6 admin admin] PATH_INFO: `/v3/projects/432f37ad628b4aedb323e795b1e3b6ab` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:35.732087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cfbc8-f573-4693-a74d-99720f84f5a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=432f37ad628b4aedb323e795b1e3b6ab)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:35.734724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cfbc8-f573-4693-a74d-99720f84f5a6 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:35.894004 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b14cfbc8-f573-4693-a74d-99720f84f5a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '432f37ad628b4aedb323e795b1e3b6ab', 'request_id': 'req-b14cfbc8-f573-4693-a74d-99720f84f5a6'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:35.899442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b14cfbc8-f573-4693-a74d-99720f84f5a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '432f37ad628b4aedb323e795b1e3b6ab', 'request_id': 'req-b14cfbc8-f573-4693-a74d-99720f84f5a6'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:35.907633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b14cfbc8-f573-4693-a74d-99720f84f5a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '432f37ad628b4aedb323e795b1e3b6ab', 'request_id': 'req-b14cfbc8-f573-4693-a74d-99720f84f5a6'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:35.989829 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b14cfbc8-f573-4693-a74d-99720f84f5a6 admin admin] The token cache is being invalidate because project 432f37ad628b4aedb323e795b1e3b6ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:35.990127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b14cfbc8-f573-4693-a74d-99720f84f5a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:35.990762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b14cfbc8-f573-4693-a74d-99720f84f5a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:35.992323 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 977/1964] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:35 2022] DELETE /identity/v3/projects/432f37ad628b4aedb323e795b1e3b6ab => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:35.997223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435f359-42c0-42a5-833e-5b227264075f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:35.997697 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435f359-42c0-42a5-833e-5b227264075f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:35.997785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435f359-42c0-42a5-833e-5b227264075f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:36.016923 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5435f359-42c0-42a5-833e-5b227264075f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:36.018219 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5435f359-42c0-42a5-833e-5b227264075f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:36.077228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 988/1965] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:36.085461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff718ac1-60ea-4ff4-abed-9ff54fff32d7 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:36.085890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff718ac1-60ea-4ff4-abed-9ff54fff32d7 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:36.086184 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff718ac1-60ea-4ff4-abed-9ff54fff32d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:36.104342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ff718ac1-60ea-4ff4-abed-9ff54fff32d7 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:36.105581 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ff718ac1-60ea-4ff4-abed-9ff54fff32d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:36.113632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 978/1966] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:36.117738 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435f359-42c0-42a5-833e-5b227264075f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:36.128181 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435f359-42c0-42a5-833e-5b227264075f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:36.132852 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391c57b-923d-4ea9-9905-f610ba763985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:36.133636 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391c57b-923d-4ea9-9905-f610ba763985 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:36.133850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391c57b-923d-4ea9-9905-f610ba763985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:36.134129 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391c57b-923d-4ea9-9905-f610ba763985 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:36.134859 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391c57b-923d-4ea9-9905-f610ba763985 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:36.135172 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391c57b-923d-4ea9-9905-f610ba763985 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:36.136761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391c57b-923d-4ea9-9905-f610ba763985 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:36.149023 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 989/1967] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:42:36.153354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff718ac1-60ea-4ff4-abed-9ff54fff32d7 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:36.163096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff718ac1-60ea-4ff4-abed-9ff54fff32d7 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:36.169503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26283f-64dd-4f90-b49e-c91ddb7a6e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:36.169503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26283f-64dd-4f90-b49e-c91ddb7a6e00 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:36.169503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26283f-64dd-4f90-b49e-c91ddb7a6e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:36.169503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26283f-64dd-4f90-b49e-c91ddb7a6e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:36.170631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26283f-64dd-4f90-b49e-c91ddb7a6e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:36.172533 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26283f-64dd-4f90-b49e-c91ddb7a6e00 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:36.269951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 979/1968] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:36 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:36.272262 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391c57b-923d-4ea9-9905-f610ba763985 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:36.283304 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391c57b-923d-4ea9-9905-f610ba763985 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:36.346391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1791f-d671-46fd-adab-e466f5f9eb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:36.347081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1791f-d671-46fd-adab-e466f5f9eb8a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:36.347458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1791f-d671-46fd-adab-e466f5f9eb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:36.347744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1791f-d671-46fd-adab-e466f5f9eb8a admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:36.348443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1791f-d671-46fd-adab-e466f5f9eb8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:36.350483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1791f-d671-46fd-adab-e466f5f9eb8a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:36.496518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 990/1969] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:36 2022] POST /identity/v3/users => generated 291 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:36.497849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26283f-64dd-4f90-b49e-c91ddb7a6e00 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:36.511434 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26283f-64dd-4f90-b49e-c91ddb7a6e00 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:36.516842 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d115f3-0451-4ad2-a35f-eb08b124c791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:36.517583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d115f3-0451-4ad2-a35f-eb08b124c791 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:36.517874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d115f3-0451-4ad2-a35f-eb08b124c791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:36.518180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d115f3-0451-4ad2-a35f-eb08b124c791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:36.518909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d115f3-0451-4ad2-a35f-eb08b124c791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:36.520527 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d115f3-0451-4ad2-a35f-eb08b124c791 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:36.532186 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 980/1970] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:36.536359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1791f-d671-46fd-adab-e466f5f9eb8a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:36.548300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1791f-d671-46fd-adab-e466f5f9eb8a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:36.554906 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca44f8-fc64-475d-aa3f-fd29ce33b1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:36.554906 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca44f8-fc64-475d-aa3f-fd29ce33b1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:36.554906 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca44f8-fc64-475d-aa3f-fd29ce33b1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:36.554906 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca44f8-fc64-475d-aa3f-fd29ce33b1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:36.555892 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca44f8-fc64-475d-aa3f-fd29ce33b1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:36.557174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca44f8-fc64-475d-aa3f-fd29ce33b1a9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:36.568458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 991/1971] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:36.572531 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d115f3-0451-4ad2-a35f-eb08b124c791 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:36.588420 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d115f3-0451-4ad2-a35f-eb08b124c791 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:36.593730 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87f355-8eff-4ce9-b2aa-c14da8275268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:36.595731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87f355-8eff-4ce9-b2aa-c14da8275268 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:36.596033 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87f355-8eff-4ce9-b2aa-c14da8275268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:36.596317 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87f355-8eff-4ce9-b2aa-c14da8275268 admin admin] PATH_INFO: `/v3/projects/268a7f3061364e81965a3727bd99b704/users/27c62279c4d74581ae9232ba69f535ff/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:36.611565 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87f355-8eff-4ce9-b2aa-c14da8275268 admin admin] RBAC: Authorizing `identity:create_grant(project_id=268a7f3061364e81965a3727bd99b704, user_id=27c62279c4d74581ae9232ba69f535ff, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:36.614289 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87f355-8eff-4ce9-b2aa-c14da8275268 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:36.722981 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 981/1972] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:36 2022] PUT /identity/v3/projects/268a7f3061364e81965a3727bd99b704/users/27c62279c4d74581ae9232ba69f535ff/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:36.744588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347f2be-7c4c-4e61-9e10-5eccdbad4c65 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:36.744884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347f2be-7c4c-4e61-9e10-5eccdbad4c65 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:36.745218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347f2be-7c4c-4e61-9e10-5eccdbad4c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:36.755209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:36.774502 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9347f2be-7c4c-4e61-9e10-5eccdbad4c65 None None] MFA Rules not processed for user `27c62279c4d74581ae9232ba69f535ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:36.775713 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9347f2be-7c4c-4e61-9e10-5eccdbad4c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:36.847483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 992/1973] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:36.851881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87f355-8eff-4ce9-b2aa-c14da8275268 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:36.935000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87f355-8eff-4ce9-b2aa-c14da8275268 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:36.940300 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40233802-0d97-46ab-8c94-753f034a702b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:36.940965 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40233802-0d97-46ab-8c94-753f034a702b admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:36.941284 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40233802-0d97-46ab-8c94-753f034a702b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:36.941568 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40233802-0d97-46ab-8c94-753f034a702b admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:36.942260 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40233802-0d97-46ab-8c94-753f034a702b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:36.944199 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40233802-0d97-46ab-8c94-753f034a702b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:36.967459 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:37.039631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 982/1974] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:36 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:37.043429 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347f2be-7c4c-4e61-9e10-5eccdbad4c65 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:37.054686 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347f2be-7c4c-4e61-9e10-5eccdbad4c65 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:37.115205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c4b15-2330-436e-b22f-e92fb2b6c035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:37.115892 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c4b15-2330-436e-b22f-e92fb2b6c035 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:37.116184 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c4b15-2330-436e-b22f-e92fb2b6c035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:37.116462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c4b15-2330-436e-b22f-e92fb2b6c035 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:37.117202 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c4b15-2330-436e-b22f-e92fb2b6c035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:37.119130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c4b15-2330-436e-b22f-e92fb2b6c035 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:37.232502 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 993/1975] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:37 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:37.237932 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40233802-0d97-46ab-8c94-753f034a702b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:37.249661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40233802-0d97-46ab-8c94-753f034a702b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:37.255495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3c5f8-9ace-413c-afaf-f8caca87aa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:37.256263 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3c5f8-9ace-413c-afaf-f8caca87aa7f admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:37.256646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3c5f8-9ace-413c-afaf-f8caca87aa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:37.257011 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3c5f8-9ace-413c-afaf-f8caca87aa7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:37.258239 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3c5f8-9ace-413c-afaf-f8caca87aa7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:37.260680 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3c5f8-9ace-413c-afaf-f8caca87aa7f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:37.277737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 983/1976] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:37.279101 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c4b15-2330-436e-b22f-e92fb2b6c035 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:37.290146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c4b15-2330-436e-b22f-e92fb2b6c035 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:37.295065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c414d-0d18-42bf-b7bb-c1a0e8bbe9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:37.295953 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c414d-0d18-42bf-b7bb-c1a0e8bbe9e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:37.296264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c414d-0d18-42bf-b7bb-c1a0e8bbe9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:37.296555 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c414d-0d18-42bf-b7bb-c1a0e8bbe9e2 admin admin] PATH_INFO: `/v3/projects/310ce774fc434a48bda37e68eb989684/users/e8adac0e6daa4d33bc33adfa582f80a8/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:37.313242 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c414d-0d18-42bf-b7bb-c1a0e8bbe9e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=310ce774fc434a48bda37e68eb989684, user_id=e8adac0e6daa4d33bc33adfa582f80a8, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:37.316002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c414d-0d18-42bf-b7bb-c1a0e8bbe9e2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:37.425874 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 994/1977] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:37 2022] PUT /identity/v3/projects/310ce774fc434a48bda37e68eb989684/users/e8adac0e6daa4d33bc33adfa582f80a8/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:37.430676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3c5f8-9ace-413c-afaf-f8caca87aa7f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:37.441657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3c5f8-9ace-413c-afaf-f8caca87aa7f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:37.482550 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:37.492215 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c414d-0d18-42bf-b7bb-c1a0e8bbe9e2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:37.494082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-de1c414d-0d18-42bf-b7bb-c1a0e8bbe9e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:37.501220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b90ab-1f24-4f6f-8980-d3c194917c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:37.502036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b90ab-1f24-4f6f-8980-d3c194917c02 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:37.502328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b90ab-1f24-4f6f-8980-d3c194917c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:37.502654 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b90ab-1f24-4f6f-8980-d3c194917c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:37.503391 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b90ab-1f24-4f6f-8980-d3c194917c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:37.505014 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b90ab-1f24-4f6f-8980-d3c194917c02 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:37.518873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 984/1978] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:37.519969 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b90ab-1f24-4f6f-8980-d3c194917c02 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:37.531459 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b90ab-1f24-4f6f-8980-d3c194917c02 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:37.536874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a4582-12a5-406d-8b85-2689f733fd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:37.537822 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a4582-12a5-406d-8b85-2689f733fd5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:37.538117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a4582-12a5-406d-8b85-2689f733fd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:37.538398 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a4582-12a5-406d-8b85-2689f733fd5e admin admin] PATH_INFO: `/v3/projects/310ce774fc434a48bda37e68eb989684/users/e8adac0e6daa4d33bc33adfa582f80a8/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:37.540393 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a4582-12a5-406d-8b85-2689f733fd5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=310ce774fc434a48bda37e68eb989684, user_id=e8adac0e6daa4d33bc33adfa582f80a8, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:37.543224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a4582-12a5-406d-8b85-2689f733fd5e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:37.555669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 985/1979] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:37 2022] PUT /identity/v3/projects/310ce774fc434a48bda37e68eb989684/users/e8adac0e6daa4d33bc33adfa582f80a8/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:37.566039 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c414d-0d18-42bf-b7bb-c1a0e8bbe9e2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:37.566376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e24ae-f78c-4322-acb5-95fa03cebbb5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:37.566684 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e24ae-f78c-4322-acb5-95fa03cebbb5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:37.566975 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e24ae-f78c-4322-acb5-95fa03cebbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:37.595921 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-174e24ae-f78c-4322-acb5-95fa03cebbb5 None None] MFA Rules not processed for user `e8adac0e6daa4d33bc33adfa582f80a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:37.597178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-174e24ae-f78c-4322-acb5-95fa03cebbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:37.627248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9004666-71d0-49d2-a388-718ef28d7a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:37.628183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9004666-71d0-49d2-a388-718ef28d7a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:37.628463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9004666-71d0-49d2-a388-718ef28d7a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:37.628739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9004666-71d0-49d2-a388-718ef28d7a9a admin admin] PATH_INFO: `/v3/users/d91414e8c7da478bb8c73f42cac64e02` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:37.629973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9004666-71d0-49d2-a388-718ef28d7a9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d91414e8c7da478bb8c73f42cac64e02)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:37.631900 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9004666-71d0-49d2-a388-718ef28d7a9a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:37.671673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 986/1980] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:37.683417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e24ae-f78c-4322-acb5-95fa03cebbb5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:37.684765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-174e24ae-f78c-4322-acb5-95fa03cebbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:37.748574 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e24ae-f78c-4322-acb5-95fa03cebbb5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:37.753723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66ee3c-d2e2-499b-abf6-d0bafa44066f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:37.754390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66ee3c-d2e2-499b-abf6-d0bafa44066f service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:37.754679 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66ee3c-d2e2-499b-abf6-d0bafa44066f service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:37.754965 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66ee3c-d2e2-499b-abf6-d0bafa44066f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:37.766473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66ee3c-d2e2-499b-abf6-d0bafa44066f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:37.767931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66ee3c-d2e2-499b-abf6-d0bafa44066f service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:37.778441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 987/1981] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:37.785995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:37.799564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d9004666-71d0-49d2-a388-718ef28d7a9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd91414e8c7da478bb8c73f42cac64e02'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:37.816293 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d9004666-71d0-49d2-a388-718ef28d7a9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd91414e8c7da478bb8c73f42cac64e02'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:37.915863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d9004666-71d0-49d2-a388-718ef28d7a9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd91414e8c7da478bb8c73f42cac64e02'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:37.916510 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d9004666-71d0-49d2-a388-718ef28d7a9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd91414e8c7da478bb8c73f42cac64e02'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:37.928901 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d9004666-71d0-49d2-a388-718ef28d7a9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd91414e8c7da478bb8c73f42cac64e02'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:37.933802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 995/1982] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:37 2022] DELETE /identity/v3/users/d91414e8c7da478bb8c73f42cac64e02 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:37.938410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc693c8f-ac2c-48a1-b991-dae72ddd3bf3 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:37.938919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc693c8f-ac2c-48a1-b991-dae72ddd3bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:37.939013 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc693c8f-ac2c-48a1-b991-dae72ddd3bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:37.950436 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:37.967849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-fc693c8f-ac2c-48a1-b991-dae72ddd3bf3 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:37.970474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fc693c8f-ac2c-48a1-b991-dae72ddd3bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:37.982603 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9004666-71d0-49d2-a388-718ef28d7a9a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:37.984262 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d9004666-71d0-49d2-a388-718ef28d7a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:38.052303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 988/1983] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:38.052453 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9004666-71d0-49d2-a388-718ef28d7a9a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:38.055138 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4162679-e07f-4be3-ad5e-86f893726cfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:38.056401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4162679-e07f-4be3-ad5e-86f893726cfb service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:38.057232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4162679-e07f-4be3-ad5e-86f893726cfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:38.057701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4162679-e07f-4be3-ad5e-86f893726cfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:38.059093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc693c8f-ac2c-48a1-b991-dae72ddd3bf3 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:38.066788 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a4162679-e07f-4be3-ad5e-86f893726cfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:38.068048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fc693c8f-ac2c-48a1-b991-dae72ddd3bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:38.131891 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4162679-e07f-4be3-ad5e-86f893726cfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:38.132492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc693c8f-ac2c-48a1-b991-dae72ddd3bf3 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:38.133381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4162679-e07f-4be3-ad5e-86f893726cfb service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:38.136148 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ccc6b-81d7-4293-b22c-efd3e7aaa794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:38.136795 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ccc6b-81d7-4293-b22c-efd3e7aaa794 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:38.137107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ccc6b-81d7-4293-b22c-efd3e7aaa794 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:38.137563 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ccc6b-81d7-4293-b22c-efd3e7aaa794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:38.142871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 996/1984] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:38.147360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ccc6b-81d7-4293-b22c-efd3e7aaa794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:38.148878 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ccc6b-81d7-4293-b22c-efd3e7aaa794 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:38.160480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 989/1985] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:38.304676 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:38.398018 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:38.666074 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4162679-e07f-4be3-ad5e-86f893726cfb service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:38.667671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a4162679-e07f-4be3-ad5e-86f893726cfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:38.685226 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4162679-e07f-4be3-ad5e-86f893726cfb service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:38.696946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bdcc7-2d24-499a-bdc4-4bbc44083bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:38.696946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bdcc7-2d24-499a-bdc4-4bbc44083bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:38.696946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bdcc7-2d24-499a-bdc4-4bbc44083bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:38.696946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bdcc7-2d24-499a-bdc4-4bbc44083bae admin admin] PATH_INFO: `/v3/projects/4c9d7a6c555f4bf4960660292e368aa4` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:38.698016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bdcc7-2d24-499a-bdc4-4bbc44083bae admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c9d7a6c555f4bf4960660292e368aa4)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:38.701850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bdcc7-2d24-499a-bdc4-4bbc44083bae admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:38.872669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:38.881423 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-745bdcc7-2d24-499a-bdc4-4bbc44083bae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c9d7a6c555f4bf4960660292e368aa4', 'request_id': 'req-745bdcc7-2d24-499a-bdc4-4bbc44083bae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:38.888508 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-745bdcc7-2d24-499a-bdc4-4bbc44083bae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c9d7a6c555f4bf4960660292e368aa4', 'request_id': 'req-745bdcc7-2d24-499a-bdc4-4bbc44083bae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:38.928633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ccc6b-81d7-4293-b22c-efd3e7aaa794 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:38.930922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-254ccc6b-81d7-4293-b22c-efd3e7aaa794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:38.948035 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ccc6b-81d7-4293-b22c-efd3e7aaa794 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:38.960972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-745bdcc7-2d24-499a-bdc4-4bbc44083bae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c9d7a6c555f4bf4960660292e368aa4', 'request_id': 'req-745bdcc7-2d24-499a-bdc4-4bbc44083bae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:38.982547 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-745bdcc7-2d24-499a-bdc4-4bbc44083bae admin admin] The token cache is being invalidate because project 4c9d7a6c555f4bf4960660292e368aa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:38.982547 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-745bdcc7-2d24-499a-bdc4-4bbc44083bae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:38.985318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-745bdcc7-2d24-499a-bdc4-4bbc44083bae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:38.987675 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 997/1986] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:38 2022] DELETE /identity/v3/projects/4c9d7a6c555f4bf4960660292e368aa4 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:38.991814 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bdcc7-2d24-499a-bdc4-4bbc44083bae admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:38.993388 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-745bdcc7-2d24-499a-bdc4-4bbc44083bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:39.046828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f598ab-0485-45bb-ae81-ab1e4043d87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:39.050638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f598ab-0485-45bb-ae81-ab1e4043d87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:39.050638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f598ab-0485-45bb-ae81-ab1e4043d87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:39.050638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f598ab-0485-45bb-ae81-ab1e4043d87c admin admin] PATH_INFO: `/v3/users/27c62279c4d74581ae9232ba69f535ff` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:39.051368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f598ab-0485-45bb-ae81-ab1e4043d87c admin admin] RBAC: Authorizing `identity:delete_user(user_id=27c62279c4d74581ae9232ba69f535ff)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:39.053841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f598ab-0485-45bb-ae81-ab1e4043d87c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:39.102722 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bdcc7-2d24-499a-bdc4-4bbc44083bae admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:39.105023 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0c990-5239-45ea-b1ea-0343f9f65a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:39.106005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0c990-5239-45ea-b1ea-0343f9f65a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:39.106342 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0c990-5239-45ea-b1ea-0343f9f65a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:39.106785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0c990-5239-45ea-b1ea-0343f9f65a75 admin admin] PATH_INFO: `/v3/users/20efc5c80a8446d5bd6507520b617036` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:39.108079 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0c990-5239-45ea-b1ea-0343f9f65a75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20efc5c80a8446d5bd6507520b617036)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:39.110209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0c990-5239-45ea-b1ea-0343f9f65a75 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:39.301844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-10f598ab-0485-45bb-ae81-ab1e4043d87c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27c62279c4d74581ae9232ba69f535ff'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:39.302706 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-10f598ab-0485-45bb-ae81-ab1e4043d87c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27c62279c4d74581ae9232ba69f535ff'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:39.303403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-10f598ab-0485-45bb-ae81-ab1e4043d87c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27c62279c4d74581ae9232ba69f535ff'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:39.313681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-10f598ab-0485-45bb-ae81-ab1e4043d87c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27c62279c4d74581ae9232ba69f535ff'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:39.317629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-65a0c990-5239-45ea-b1ea-0343f9f65a75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20efc5c80a8446d5bd6507520b617036'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:39.327233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-65a0c990-5239-45ea-b1ea-0343f9f65a75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20efc5c80a8446d5bd6507520b617036'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:39.329390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-10f598ab-0485-45bb-ae81-ab1e4043d87c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27c62279c4d74581ae9232ba69f535ff'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:39.379033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-65a0c990-5239-45ea-b1ea-0343f9f65a75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20efc5c80a8446d5bd6507520b617036'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:39.379687 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-65a0c990-5239-45ea-b1ea-0343f9f65a75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20efc5c80a8446d5bd6507520b617036'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:39.387034 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-65a0c990-5239-45ea-b1ea-0343f9f65a75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20efc5c80a8446d5bd6507520b617036'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:39.393534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 998/1987] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:38 2022] DELETE /identity/v3/users/20efc5c80a8446d5bd6507520b617036 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:39.413786 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:39.446359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 990/1988] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:38 2022] DELETE /identity/v3/users/27c62279c4d74581ae9232ba69f535ff => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:39.450984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cd9b4-3655-4cd0-85a7-24a5349ae97e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:39.451339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cd9b4-3655-4cd0-85a7-24a5349ae97e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:39.451637 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cd9b4-3655-4cd0-85a7-24a5349ae97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:39.476119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-802cd9b4-3655-4cd0-85a7-24a5349ae97e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:39.477381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-802cd9b4-3655-4cd0-85a7-24a5349ae97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:39.550156 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 999/1989] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:39.559855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f598ab-0485-45bb-ae81-ab1e4043d87c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:39.561275 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-10f598ab-0485-45bb-ae81-ab1e4043d87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:39.626073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f598ab-0485-45bb-ae81-ab1e4043d87c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:39.631015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bef047-3637-4d3b-8d31-cab1f8b08771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:39.631890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bef047-3637-4d3b-8d31-cab1f8b08771 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:39.633083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bef047-3637-4d3b-8d31-cab1f8b08771 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:39.633461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bef047-3637-4d3b-8d31-cab1f8b08771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:39.645320 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bef047-3637-4d3b-8d31-cab1f8b08771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:39.646832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bef047-3637-4d3b-8d31-cab1f8b08771 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:39.657352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 991/1990] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:39.870698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:39.906249 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802cd9b4-3655-4cd0-85a7-24a5349ae97e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:39.908253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-802cd9b4-3655-4cd0-85a7-24a5349ae97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:39.920683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802cd9b4-3655-4cd0-85a7-24a5349ae97e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:39.928995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427cab9-cb27-4af2-9661-379b78072a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:39.930150 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427cab9-cb27-4af2-9661-379b78072a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:39.930502 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427cab9-cb27-4af2-9661-379b78072a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:39.930785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427cab9-cb27-4af2-9661-379b78072a68 admin admin] PATH_INFO: `/v3/projects/a09b0ba895f94450b2fe5f3d3a358de7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:39.932566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427cab9-cb27-4af2-9661-379b78072a68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a09b0ba895f94450b2fe5f3d3a358de7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:39.934576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427cab9-cb27-4af2-9661-379b78072a68 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:40.046745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b427cab9-cb27-4af2-9661-379b78072a68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a09b0ba895f94450b2fe5f3d3a358de7', 'request_id': 'req-b427cab9-cb27-4af2-9661-379b78072a68'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.055613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b427cab9-cb27-4af2-9661-379b78072a68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a09b0ba895f94450b2fe5f3d3a358de7', 'request_id': 'req-b427cab9-cb27-4af2-9661-379b78072a68'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.207132 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b427cab9-cb27-4af2-9661-379b78072a68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a09b0ba895f94450b2fe5f3d3a358de7', 'request_id': 'req-b427cab9-cb27-4af2-9661-379b78072a68'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.218422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b427cab9-cb27-4af2-9661-379b78072a68 admin admin] The token cache is being invalidate because project a09b0ba895f94450b2fe5f3d3a358de7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:40.218654 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b427cab9-cb27-4af2-9661-379b78072a68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.219394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b427cab9-cb27-4af2-9661-379b78072a68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.220969 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1000/1991] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:39 2022] DELETE /identity/v3/projects/a09b0ba895f94450b2fe5f3d3a358de7 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:40.225011 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bef047-3637-4d3b-8d31-cab1f8b08771 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:40.235475 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bef047-3637-4d3b-8d31-cab1f8b08771 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:40.261134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427cab9-cb27-4af2-9661-379b78072a68 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:40.264887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b427cab9-cb27-4af2-9661-379b78072a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:40.321141 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a076c2-4a93-4fc7-be8f-acf7626e7fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:40.322672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a076c2-4a93-4fc7-be8f-acf7626e7fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:40.323142 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a076c2-4a93-4fc7-be8f-acf7626e7fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:40.323426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a076c2-4a93-4fc7-be8f-acf7626e7fef admin admin] PATH_INFO: `/v3/users/a6f8996eab144e52906943c68114ed09` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:40.324712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a076c2-4a93-4fc7-be8f-acf7626e7fef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f8996eab144e52906943c68114ed09)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:40.326931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a076c2-4a93-4fc7-be8f-acf7626e7fef admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:40.368932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427cab9-cb27-4af2-9661-379b78072a68 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:40.374012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:40.374885 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:40.375197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:40.375591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c admin admin] PATH_INFO: `/v3/projects/268a7f3061364e81965a3727bd99b704` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:40.376923 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c admin admin] RBAC: Authorizing `identity:delete_project(project_id=268a7f3061364e81965a3727bd99b704)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:40.378882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:40.496761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:40.508183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '268a7f3061364e81965a3727bd99b704', 'request_id': 'req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.510924 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e9a076c2-4a93-4fc7-be8f-acf7626e7fef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f8996eab144e52906943c68114ed09'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.511552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e9a076c2-4a93-4fc7-be8f-acf7626e7fef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f8996eab144e52906943c68114ed09'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.512163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e9a076c2-4a93-4fc7-be8f-acf7626e7fef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f8996eab144e52906943c68114ed09'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.514771 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '268a7f3061364e81965a3727bd99b704', 'request_id': 'req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.519246 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e9a076c2-4a93-4fc7-be8f-acf7626e7fef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f8996eab144e52906943c68114ed09'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.530744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e9a076c2-4a93-4fc7-be8f-acf7626e7fef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f8996eab144e52906943c68114ed09'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.596006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '268a7f3061364e81965a3727bd99b704', 'request_id': 'req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.606920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c admin admin] The token cache is being invalidate because project 268a7f3061364e81965a3727bd99b704 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:40.607214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.607863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.609484 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1001/1992] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:40 2022] DELETE /identity/v3/projects/268a7f3061364e81965a3727bd99b704 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:40.614254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:40.615611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:40.676284 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 992/1993] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:40 2022] DELETE /identity/v3/users/a6f8996eab144e52906943c68114ed09 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:40.680203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7b606-19e2-4a68-89bf-b8a5cb3f193c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:40.739784 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad05fc-568a-4eaf-8d71-08dab655eb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:40.740723 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad05fc-568a-4eaf-8d71-08dab655eb0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:40.741041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad05fc-568a-4eaf-8d71-08dab655eb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:40.741353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad05fc-568a-4eaf-8d71-08dab655eb0e admin admin] PATH_INFO: `/v3/users/e8adac0e6daa4d33bc33adfa582f80a8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:40.742587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad05fc-568a-4eaf-8d71-08dab655eb0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8adac0e6daa4d33bc33adfa582f80a8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:40.744745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad05fc-568a-4eaf-8d71-08dab655eb0e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:40.921348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2fad05fc-568a-4eaf-8d71-08dab655eb0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8adac0e6daa4d33bc33adfa582f80a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.933888 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2fad05fc-568a-4eaf-8d71-08dab655eb0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8adac0e6daa4d33bc33adfa582f80a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.989382 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2fad05fc-568a-4eaf-8d71-08dab655eb0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8adac0e6daa4d33bc33adfa582f80a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.990083 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2fad05fc-568a-4eaf-8d71-08dab655eb0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8adac0e6daa4d33bc33adfa582f80a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:40.999539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2fad05fc-568a-4eaf-8d71-08dab655eb0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8adac0e6daa4d33bc33adfa582f80a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:41.007053 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1002/1994] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:40 2022] DELETE /identity/v3/users/e8adac0e6daa4d33bc33adfa582f80a8 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:41.138885 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a076c2-4a93-4fc7-be8f-acf7626e7fef admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:41.140370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e9a076c2-4a93-4fc7-be8f-acf7626e7fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:41.153947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a076c2-4a93-4fc7-be8f-acf7626e7fef admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:41.220498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1ac19-55ac-4634-8ad3-7cd7013347f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:41.220498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1ac19-55ac-4634-8ad3-7cd7013347f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:41.220498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1ac19-55ac-4634-8ad3-7cd7013347f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:41.220498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1ac19-55ac-4634-8ad3-7cd7013347f1 admin admin] PATH_INFO: `/v3/projects/156e936402d04354a66c9ce917949001` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:41.220912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1ac19-55ac-4634-8ad3-7cd7013347f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=156e936402d04354a66c9ce917949001)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:41.222296 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1ac19-55ac-4634-8ad3-7cd7013347f1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:41.299822 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:41.342447 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-62e1ac19-55ac-4634-8ad3-7cd7013347f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '156e936402d04354a66c9ce917949001', 'request_id': 'req-62e1ac19-55ac-4634-8ad3-7cd7013347f1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:41.349150 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-62e1ac19-55ac-4634-8ad3-7cd7013347f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '156e936402d04354a66c9ce917949001', 'request_id': 'req-62e1ac19-55ac-4634-8ad3-7cd7013347f1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:41.356110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-62e1ac19-55ac-4634-8ad3-7cd7013347f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '156e936402d04354a66c9ce917949001', 'request_id': 'req-62e1ac19-55ac-4634-8ad3-7cd7013347f1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:41.430043 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-62e1ac19-55ac-4634-8ad3-7cd7013347f1 admin admin] The token cache is being invalidate because project 156e936402d04354a66c9ce917949001 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:41.430346 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-62e1ac19-55ac-4634-8ad3-7cd7013347f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:41.431157 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-62e1ac19-55ac-4634-8ad3-7cd7013347f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:41.434449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 993/1995] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:41 2022] DELETE /identity/v3/projects/156e936402d04354a66c9ce917949001 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:41.440189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9ea11-b75d-4213-89f3-c0912f3ba6a2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:41.440479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9ea11-b75d-4213-89f3-c0912f3ba6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:41.440755 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9ea11-b75d-4213-89f3-c0912f3ba6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:41.449247 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:41.459936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-09a9ea11-b75d-4213-89f3-c0912f3ba6a2 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:41.465488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-09a9ea11-b75d-4213-89f3-c0912f3ba6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:41.558093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1003/1996] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:41.567875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1eea78-ef4e-4955-8f8d-1a624c33d977 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:41.568169 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1eea78-ef4e-4955-8f8d-1a624c33d977 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:41.568453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1eea78-ef4e-4955-8f8d-1a624c33d977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:41.587612 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-de1eea78-ef4e-4955-8f8d-1a624c33d977 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:41.588846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-de1eea78-ef4e-4955-8f8d-1a624c33d977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:41.598318 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 994/1997] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:41.607373 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9ea11-b75d-4213-89f3-c0912f3ba6a2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:41.617647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9ea11-b75d-4213-89f3-c0912f3ba6a2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:41.623052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2eada-29e8-491c-8bf5-fa1a849740e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:41.623733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2eada-29e8-491c-8bf5-fa1a849740e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:41.624020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2eada-29e8-491c-8bf5-fa1a849740e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:41.624307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2eada-29e8-491c-8bf5-fa1a849740e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:41.625046 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2eada-29e8-491c-8bf5-fa1a849740e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:41.626066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2eada-29e8-491c-8bf5-fa1a849740e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:41.627681 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2eada-29e8-491c-8bf5-fa1a849740e9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:41.642307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1004/1998] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:42:41.648441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1eea78-ef4e-4955-8f8d-1a624c33d977 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:41.658916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1eea78-ef4e-4955-8f8d-1a624c33d977 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:41.663930 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75baeffc-da77-4e8c-88e1-968cdb54acb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:41.664649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75baeffc-da77-4e8c-88e1-968cdb54acb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:41.664874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75baeffc-da77-4e8c-88e1-968cdb54acb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:41.665187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75baeffc-da77-4e8c-88e1-968cdb54acb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:41.667534 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75baeffc-da77-4e8c-88e1-968cdb54acb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:41.667990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75baeffc-da77-4e8c-88e1-968cdb54acb1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:41.671910 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2eada-29e8-491c-8bf5-fa1a849740e9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:41.674409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cea2eada-29e8-491c-8bf5-fa1a849740e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:41.697909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2eada-29e8-491c-8bf5-fa1a849740e9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:41.706844 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02722b8-de81-47a9-b01e-b19f944f2dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:41.707746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02722b8-de81-47a9-b01e-b19f944f2dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:41.707967 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02722b8-de81-47a9-b01e-b19f944f2dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:41.708371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02722b8-de81-47a9-b01e-b19f944f2dc1 admin admin] PATH_INFO: `/v3/projects/310ce774fc434a48bda37e68eb989684` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:41.709748 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02722b8-de81-47a9-b01e-b19f944f2dc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=310ce774fc434a48bda37e68eb989684)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:41.711650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02722b8-de81-47a9-b01e-b19f944f2dc1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:41.777414 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 995/1999] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:41 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:41.777414 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75baeffc-da77-4e8c-88e1-968cdb54acb1 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:41.777414 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-75baeffc-da77-4e8c-88e1-968cdb54acb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:41.840393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c02722b8-de81-47a9-b01e-b19f944f2dc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '310ce774fc434a48bda37e68eb989684', 'request_id': 'req-c02722b8-de81-47a9-b01e-b19f944f2dc1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:41.841770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75baeffc-da77-4e8c-88e1-968cdb54acb1 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:41.849577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c02722b8-de81-47a9-b01e-b19f944f2dc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '310ce774fc434a48bda37e68eb989684', 'request_id': 'req-c02722b8-de81-47a9-b01e-b19f944f2dc1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:41.902247 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b296bc-e063-46a5-b91d-5df3aef8ab20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:41.903183 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b296bc-e063-46a5-b91d-5df3aef8ab20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:41.903471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b296bc-e063-46a5-b91d-5df3aef8ab20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:41.903755 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b296bc-e063-46a5-b91d-5df3aef8ab20 admin admin] PATH_INFO: `/v3/users/7b83a14b8fcc4492b37c4da4db8edeff` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:41.904969 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b296bc-e063-46a5-b91d-5df3aef8ab20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b83a14b8fcc4492b37c4da4db8edeff)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:41.907565 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b296bc-e063-46a5-b91d-5df3aef8ab20 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:41.915795 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c02722b8-de81-47a9-b01e-b19f944f2dc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '310ce774fc434a48bda37e68eb989684', 'request_id': 'req-c02722b8-de81-47a9-b01e-b19f944f2dc1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:41.927778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c02722b8-de81-47a9-b01e-b19f944f2dc1 admin admin] The token cache is being invalidate because project 310ce774fc434a48bda37e68eb989684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:41.928069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c02722b8-de81-47a9-b01e-b19f944f2dc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:41.928713 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c02722b8-de81-47a9-b01e-b19f944f2dc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:41.930551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1005/2000] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:41 2022] DELETE /identity/v3/projects/310ce774fc434a48bda37e68eb989684 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:41.931319 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02722b8-de81-47a9-b01e-b19f944f2dc1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:41.932592 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c02722b8-de81-47a9-b01e-b19f944f2dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:41.945499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02722b8-de81-47a9-b01e-b19f944f2dc1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:41.950171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805d364-94a5-4cd5-a97f-34c545804688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:41.950824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805d364-94a5-4cd5-a97f-34c545804688 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:41.951105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805d364-94a5-4cd5-a97f-34c545804688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:41.951381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805d364-94a5-4cd5-a97f-34c545804688 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:41.952064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805d364-94a5-4cd5-a97f-34c545804688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:41.954013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805d364-94a5-4cd5-a97f-34c545804688 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:42.042035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1006/2001] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:41 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:42.043426 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ea13c-352a-432e-9c15-8545c8ea68c8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.043718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ea13c-352a-432e-9c15-8545c8ea68c8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.043998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ea13c-352a-432e-9c15-8545c8ea68c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.061926 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-102ea13c-352a-432e-9c15-8545c8ea68c8 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:42.063178 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-102ea13c-352a-432e-9c15-8545c8ea68c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:42.070924 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1007/2002] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:42.071675 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ea13c-352a-432e-9c15-8545c8ea68c8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:42.080868 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ea13c-352a-432e-9c15-8545c8ea68c8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:42.085694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20feb22c-66a9-48a7-be24-21425ecf7776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:42.086347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20feb22c-66a9-48a7-be24-21425ecf7776 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.086638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20feb22c-66a9-48a7-be24-21425ecf7776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.086925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20feb22c-66a9-48a7-be24-21425ecf7776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.087652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20feb22c-66a9-48a7-be24-21425ecf7776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:42.089270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20feb22c-66a9-48a7-be24-21425ecf7776 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:42.100004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1008/2003] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:42.101131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9d829-95c8-4841-b551-c8ed217b962d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.101423 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9d829-95c8-4841-b551-c8ed217b962d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.101756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9d829-95c8-4841-b551-c8ed217b962d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.120205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-88f9d829-95c8-4841-b551-c8ed217b962d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:42.121463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-88f9d829-95c8-4841-b551-c8ed217b962d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:42.130091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1009/2004] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:42.133982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9d829-95c8-4841-b551-c8ed217b962d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:42.146176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9d829-95c8-4841-b551-c8ed217b962d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:42.147056 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52b296bc-e063-46a5-b91d-5df3aef8ab20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b83a14b8fcc4492b37c4da4db8edeff'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:42.147816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52b296bc-e063-46a5-b91d-5df3aef8ab20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b83a14b8fcc4492b37c4da4db8edeff'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:42.148545 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52b296bc-e063-46a5-b91d-5df3aef8ab20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b83a14b8fcc4492b37c4da4db8edeff'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:42.151899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd918bd-3ca6-461c-bd1a-d2526ec6205c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:42.152598 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd918bd-3ca6-461c-bd1a-d2526ec6205c admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.152888 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd918bd-3ca6-461c-bd1a-d2526ec6205c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.153199 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd918bd-3ca6-461c-bd1a-d2526ec6205c admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.154095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd918bd-3ca6-461c-bd1a-d2526ec6205c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:42.155556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd918bd-3ca6-461c-bd1a-d2526ec6205c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:42.164997 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52b296bc-e063-46a5-b91d-5df3aef8ab20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b83a14b8fcc4492b37c4da4db8edeff'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:42.167803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1010/2005] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:42.170263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd918bd-3ca6-461c-bd1a-d2526ec6205c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:42.174304 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8bd918bd-3ca6-461c-bd1a-d2526ec6205c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:42.180016 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52b296bc-e063-46a5-b91d-5df3aef8ab20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b83a14b8fcc4492b37c4da4db8edeff'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:42.188763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd918bd-3ca6-461c-bd1a-d2526ec6205c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:42.193333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb54aa-26ca-4772-8ad3-83f6c9b279fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:42.194096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb54aa-26ca-4772-8ad3-83f6c9b279fd admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.194387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb54aa-26ca-4772-8ad3-83f6c9b279fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.194703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb54aa-26ca-4772-8ad3-83f6c9b279fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.195981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb54aa-26ca-4772-8ad3-83f6c9b279fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:42.196484 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb54aa-26ca-4772-8ad3-83f6c9b279fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:42.199152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb54aa-26ca-4772-8ad3-83f6c9b279fd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:42.217715 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1011/2006] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:42:42.218737 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb54aa-26ca-4772-8ad3-83f6c9b279fd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:42.221611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-98eb54aa-26ca-4772-8ad3-83f6c9b279fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:42.234052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb54aa-26ca-4772-8ad3-83f6c9b279fd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:42.239686 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8b779-305d-48dc-b933-9454958e0283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:42.240569 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8b779-305d-48dc-b933-9454958e0283 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.240857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8b779-305d-48dc-b933-9454958e0283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.241171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8b779-305d-48dc-b933-9454958e0283 admin admin] PATH_INFO: `/v3/projects/b8ef3f1e3f384c4cb3c4c267a5365c06/users/27ed4d43434c448a9f890d2965a606c2/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.255288 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8b779-305d-48dc-b933-9454958e0283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ef3f1e3f384c4cb3c4c267a5365c06, user_id=27ed4d43434c448a9f890d2965a606c2, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:42.257988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8b779-305d-48dc-b933-9454958e0283 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:42.275750 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 996/2007] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:41 2022] DELETE /identity/v3/users/7b83a14b8fcc4492b37c4da4db8edeff => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:42.276760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b296bc-e063-46a5-b91d-5df3aef8ab20 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:42.337113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b296bc-e063-46a5-b91d-5df3aef8ab20 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:42.341721 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a054d0a-f174-496f-ba9d-e5d757f2cbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:42.342411 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a054d0a-f174-496f-ba9d-e5d757f2cbc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.342764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a054d0a-f174-496f-ba9d-e5d757f2cbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.343305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a054d0a-f174-496f-ba9d-e5d757f2cbc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.343733 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a054d0a-f174-496f-ba9d-e5d757f2cbc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:42.346280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a054d0a-f174-496f-ba9d-e5d757f2cbc3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:42.369487 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1012/2008] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:42 2022] PUT /identity/v3/projects/b8ef3f1e3f384c4cb3c4c267a5365c06/users/27ed4d43434c448a9f890d2965a606c2/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:42.369487 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea9fa2-2565-451e-b203-29b4f5fc3a2d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.369487 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea9fa2-2565-451e-b203-29b4f5fc3a2d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.369487 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea9fa2-2565-451e-b203-29b4f5fc3a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.382824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d9ea9fa2-2565-451e-b203-29b4f5fc3a2d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:42.384089 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d9ea9fa2-2565-451e-b203-29b4f5fc3a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:42.441599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1013/2009] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:42.443045 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a75cb6-ce12-4cb2-bacf-e0c55593e730 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.443338 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a75cb6-ce12-4cb2-bacf-e0c55593e730 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.443621 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a75cb6-ce12-4cb2-bacf-e0c55593e730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.444131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 997/2010] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:42 2022] POST /identity/v3/projects => generated 361 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:42.448064 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a054d0a-f174-496f-ba9d-e5d757f2cbc3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:42.450581 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:42.476703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-50a75cb6-ce12-4cb2-bacf-e0c55593e730 None None] MFA Rules not processed for user `27ed4d43434c448a9f890d2965a606c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:42.477942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-50a75cb6-ce12-4cb2-bacf-e0c55593e730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:42.514775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a054d0a-f174-496f-ba9d-e5d757f2cbc3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:42.520519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca1fd8-5aea-48a9-97ad-53f69e3014aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:42.521236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca1fd8-5aea-48a9-97ad-53f69e3014aa admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.521528 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca1fd8-5aea-48a9-97ad-53f69e3014aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.521812 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca1fd8-5aea-48a9-97ad-53f69e3014aa admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.522508 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca1fd8-5aea-48a9-97ad-53f69e3014aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:42.524494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca1fd8-5aea-48a9-97ad-53f69e3014aa admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:42.535623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:42.551952 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1014/2011] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:42.553003 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a75cb6-ce12-4cb2-bacf-e0c55593e730 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:42.554486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-50a75cb6-ce12-4cb2-bacf-e0c55593e730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:42.588585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 998/2012] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:42 2022] POST /identity/v3/users => generated 281 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:42.588585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca1fd8-5aea-48a9-97ad-53f69e3014aa admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:42.597831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca1fd8-5aea-48a9-97ad-53f69e3014aa admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:42.603502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efaadf-57ec-44a0-b27b-e3a373dbd204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:42.604330 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efaadf-57ec-44a0-b27b-e3a373dbd204 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.604688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efaadf-57ec-44a0-b27b-e3a373dbd204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.605211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efaadf-57ec-44a0-b27b-e3a373dbd204 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.606149 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efaadf-57ec-44a0-b27b-e3a373dbd204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:42.608145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efaadf-57ec-44a0-b27b-e3a373dbd204 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:42.628379 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a75cb6-ce12-4cb2-bacf-e0c55593e730 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:42.632206 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038979a2-548a-4e1c-badc-d705d895ace8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:42.636566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038979a2-548a-4e1c-badc-d705d895ace8 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.636566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038979a2-548a-4e1c-badc-d705d895ace8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.636566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038979a2-548a-4e1c-badc-d705d895ace8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.642993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038979a2-548a-4e1c-badc-d705d895ace8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:42.644422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038979a2-548a-4e1c-badc-d705d895ace8 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:42.654019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1015/2013] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:42.654834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038979a2-548a-4e1c-badc-d705d895ace8 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:42.665323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038979a2-548a-4e1c-badc-d705d895ace8 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:42.671354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022888a3-892e-490c-8217-d007956434f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:42.671613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022888a3-892e-490c-8217-d007956434f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.671836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022888a3-892e-490c-8217-d007956434f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.672218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022888a3-892e-490c-8217-d007956434f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.673469 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022888a3-892e-490c-8217-d007956434f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:42.675065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022888a3-892e-490c-8217-d007956434f5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:42.692617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1016/2014] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:42.695444 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022888a3-892e-490c-8217-d007956434f5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:42.700003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 999/2015] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:42 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:42.704677 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efaadf-57ec-44a0-b27b-e3a373dbd204 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:42.709124 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022888a3-892e-490c-8217-d007956434f5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:42.788590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efaadf-57ec-44a0-b27b-e3a373dbd204 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:42.792035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef18491-5817-40bb-9199-26ace4450ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:42.792727 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef18491-5817-40bb-9199-26ace4450ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.793021 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef18491-5817-40bb-9199-26ace4450ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.793337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef18491-5817-40bb-9199-26ace4450ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.793741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81927a-8daf-4b07-b5f0-2c717f7c8a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:42.794326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef18491-5817-40bb-9199-26ace4450ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:42.794768 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81927a-8daf-4b07-b5f0-2c717f7c8a23 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.795053 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81927a-8daf-4b07-b5f0-2c717f7c8a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.795356 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81927a-8daf-4b07-b5f0-2c717f7c8a23 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.795924 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef18491-5817-40bb-9199-26ace4450ea1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:42.796040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81927a-8daf-4b07-b5f0-2c717f7c8a23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:42.798844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81927a-8daf-4b07-b5f0-2c717f7c8a23 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:42.806757 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1017/2016] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:42.809938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:42.812321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef18491-5817-40bb-9199-26ace4450ea1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:42.821665 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef18491-5817-40bb-9199-26ace4450ea1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:42.826468 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf3215-3a79-4c50-87b7-e50637642e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:42.827368 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf3215-3a79-4c50-87b7-e50637642e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:42.827659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf3215-3a79-4c50-87b7-e50637642e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:42.827945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf3215-3a79-4c50-87b7-e50637642e43 admin admin] PATH_INFO: `/v3/projects/8f87b19a30fe43bfbc9b5e2ff6dee915/users/6636e0df1ead4aa68f6386ce6965556b/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:42.846559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf3215-3a79-4c50-87b7-e50637642e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f87b19a30fe43bfbc9b5e2ff6dee915, user_id=6636e0df1ead4aa68f6386ce6965556b, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:42.849327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf3215-3a79-4c50-87b7-e50637642e43 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:42.986620 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1000/2017] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:42 2022] POST /identity/v3/users => generated 294 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:42.990433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81927a-8daf-4b07-b5f0-2c717f7c8a23 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:43.000537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81927a-8daf-4b07-b5f0-2c717f7c8a23 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:43.005513 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c4b12-88ab-48c7-9edb-d84d5e79b854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:43.006173 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c4b12-88ab-48c7-9edb-d84d5e79b854 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:43.006457 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c4b12-88ab-48c7-9edb-d84d5e79b854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:43.006738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c4b12-88ab-48c7-9edb-d84d5e79b854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:43.007459 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c4b12-88ab-48c7-9edb-d84d5e79b854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:43.009076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c4b12-88ab-48c7-9edb-d84d5e79b854 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:43.009193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1018/2018] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:42 2022] PUT /identity/v3/projects/8f87b19a30fe43bfbc9b5e2ff6dee915/users/6636e0df1ead4aa68f6386ce6965556b/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:43.021217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153225d2-5d94-4c0b-bc9f-eaf72f488d28 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:43.021589 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153225d2-5d94-4c0b-bc9f-eaf72f488d28 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:43.021945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153225d2-5d94-4c0b-bc9f-eaf72f488d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:43.026251 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1001/2019] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:43.027220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c4b12-88ab-48c7-9edb-d84d5e79b854 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:43.042239 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c4b12-88ab-48c7-9edb-d84d5e79b854 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:43.050000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-153225d2-5d94-4c0b-bc9f-eaf72f488d28 None None] MFA Rules not processed for user `6636e0df1ead4aa68f6386ce6965556b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:43.051241 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-153225d2-5d94-4c0b-bc9f-eaf72f488d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:43.108668 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba2615-2a1d-4f29-b67b-716a378a9403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:43.110358 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba2615-2a1d-4f29-b67b-716a378a9403 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:43.110827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba2615-2a1d-4f29-b67b-716a378a9403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:43.111114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba2615-2a1d-4f29-b67b-716a378a9403 admin admin] PATH_INFO: `/v3/projects/38fd6f23aff249658b7b883c9bea9984/users/93a5a8ffb89245629058eac80dd9aaf9/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:43.127653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba2615-2a1d-4f29-b67b-716a378a9403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38fd6f23aff249658b7b883c9bea9984, user_id=93a5a8ffb89245629058eac80dd9aaf9, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:43.130758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba2615-2a1d-4f29-b67b-716a378a9403 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:43.151608 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1019/2020] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:43.152401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153225d2-5d94-4c0b-bc9f-eaf72f488d28 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:43.153891 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-153225d2-5d94-4c0b-bc9f-eaf72f488d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:43.172632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153225d2-5d94-4c0b-bc9f-eaf72f488d28 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:43.182765 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:43.185074 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:43.186371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:43.186645 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:43.186993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2 admin admin] PATH_INFO: `/v3/projects/c892b37d74de4f8eae6be3871be771d3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:43.188229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c892b37d74de4f8eae6be3871be771d3)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:43.190581 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:43.205772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1002/2021] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:43 2022] PUT /identity/v3/projects/38fd6f23aff249658b7b883c9bea9984/users/93a5a8ffb89245629058eac80dd9aaf9/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:43.205772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba2615-2a1d-4f29-b67b-716a378a9403 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:43.205772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b5ba2615-2a1d-4f29-b67b-716a378a9403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:43.274676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba2615-2a1d-4f29-b67b-716a378a9403 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:43.278393 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d119cc1-ac44-47e3-988d-bd3436098a48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:43.279052 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d119cc1-ac44-47e3-988d-bd3436098a48 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:43.279352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d119cc1-ac44-47e3-988d-bd3436098a48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:43.280002 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d119cc1-ac44-47e3-988d-bd3436098a48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:43.303332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c892b37d74de4f8eae6be3871be771d3', 'request_id': 'req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:43.310285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c892b37d74de4f8eae6be3871be771d3', 'request_id': 'req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:43.349047 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d119cc1-ac44-47e3-988d-bd3436098a48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:43.350551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d119cc1-ac44-47e3-988d-bd3436098a48 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:43.359191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1003/2022] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:43.360119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d119cc1-ac44-47e3-988d-bd3436098a48 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:43.397605 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c892b37d74de4f8eae6be3871be771d3', 'request_id': 'req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:43.409272 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2 admin admin] The token cache is being invalidate because project c892b37d74de4f8eae6be3871be771d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:43.409480 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:43.410202 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:43.415218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1020/2023] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:43 2022] DELETE /identity/v3/projects/c892b37d74de4f8eae6be3871be771d3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:43.416188 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:43.417927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:43.432065 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d119cc1-ac44-47e3-988d-bd3436098a48 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:43.469821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:43.492322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab47a0-416c-4ec3-a2b9-29e2c5dd9ea2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:43.494485 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95b91e-eab5-4080-a810-15657d93b913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:43.495168 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95b91e-eab5-4080-a810-15657d93b913 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:43.495466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95b91e-eab5-4080-a810-15657d93b913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:43.495794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95b91e-eab5-4080-a810-15657d93b913 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:43.496524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95b91e-eab5-4080-a810-15657d93b913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:43.497080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb1d89-c9f4-4500-973d-bc434a9ee5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:43.500972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb1d89-c9f4-4500-973d-bc434a9ee5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:43.501105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95b91e-eab5-4080-a810-15657d93b913 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:43.501197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb1d89-c9f4-4500-973d-bc434a9ee5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:43.501197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb1d89-c9f4-4500-973d-bc434a9ee5f0 admin admin] PATH_INFO: `/v3/users/6636e0df1ead4aa68f6386ce6965556b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:43.501197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb1d89-c9f4-4500-973d-bc434a9ee5f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6636e0df1ead4aa68f6386ce6965556b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:43.502410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb1d89-c9f4-4500-973d-bc434a9ee5f0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:43.509266 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1004/2024] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:43.510093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95b91e-eab5-4080-a810-15657d93b913 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:43.511819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-aa95b91e-eab5-4080-a810-15657d93b913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:43.525573 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95b91e-eab5-4080-a810-15657d93b913 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:43.530385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0cc06-f5d3-4dc5-ba35-f0ab8a40c2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:43.531289 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0cc06-f5d3-4dc5-ba35-f0ab8a40c2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:43.531648 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0cc06-f5d3-4dc5-ba35-f0ab8a40c2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:43.532052 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0cc06-f5d3-4dc5-ba35-f0ab8a40c2cb admin admin] PATH_INFO: `/v3/users/e206013b6cb04bb2ac8522f44d507393` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:43.533424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0cc06-f5d3-4dc5-ba35-f0ab8a40c2cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e206013b6cb04bb2ac8522f44d507393)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:43.535386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0cc06-f5d3-4dc5-ba35-f0ab8a40c2cb admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:43.723347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-41cb1d89-c9f4-4500-973d-bc434a9ee5f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6636e0df1ead4aa68f6386ce6965556b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:43.737042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-41cb1d89-c9f4-4500-973d-bc434a9ee5f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6636e0df1ead4aa68f6386ce6965556b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:43.739825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b7c0cc06-f5d3-4dc5-ba35-f0ab8a40c2cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e206013b6cb04bb2ac8522f44d507393'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:43.740420 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b7c0cc06-f5d3-4dc5-ba35-f0ab8a40c2cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e206013b6cb04bb2ac8522f44d507393'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:43.741120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b7c0cc06-f5d3-4dc5-ba35-f0ab8a40c2cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e206013b6cb04bb2ac8522f44d507393'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:43.748633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b7c0cc06-f5d3-4dc5-ba35-f0ab8a40c2cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e206013b6cb04bb2ac8522f44d507393'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:43.762893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b7c0cc06-f5d3-4dc5-ba35-f0ab8a40c2cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e206013b6cb04bb2ac8522f44d507393'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:43.773228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:43.790252 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-41cb1d89-c9f4-4500-973d-bc434a9ee5f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6636e0df1ead4aa68f6386ce6965556b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:43.790938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-41cb1d89-c9f4-4500-973d-bc434a9ee5f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6636e0df1ead4aa68f6386ce6965556b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:43.799465 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-41cb1d89-c9f4-4500-973d-bc434a9ee5f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6636e0df1ead4aa68f6386ce6965556b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:43.805818 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1021/2025] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:43 2022] DELETE /identity/v3/users/6636e0df1ead4aa68f6386ce6965556b => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:43.806664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb1d89-c9f4-4500-973d-bc434a9ee5f0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:43.807985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-41cb1d89-c9f4-4500-973d-bc434a9ee5f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:43.850821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1005/2026] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:43 2022] DELETE /identity/v3/users/e206013b6cb04bb2ac8522f44d507393 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:43.850821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e437b9e-9def-4682-9ac6-bc108b573d9f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:43.850821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e437b9e-9def-4682-9ac6-bc108b573d9f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:43.850821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e437b9e-9def-4682-9ac6-bc108b573d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:43.878439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3e437b9e-9def-4682-9ac6-bc108b573d9f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:43.888029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3e437b9e-9def-4682-9ac6-bc108b573d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:43.903173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb1d89-c9f4-4500-973d-bc434a9ee5f0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:43.965218 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1006/2027] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:43.974990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e437b9e-9def-4682-9ac6-bc108b573d9f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:43.976766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3e437b9e-9def-4682-9ac6-bc108b573d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:43.994149 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164f892-af0d-427d-90c8-0c2f7d24125b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:43.995048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164f892-af0d-427d-90c8-0c2f7d24125b admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:43.995337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164f892-af0d-427d-90c8-0c2f7d24125b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:43.995612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164f892-af0d-427d-90c8-0c2f7d24125b admin admin] PATH_INFO: `/v3/projects/38fd6f23aff249658b7b883c9bea9984/users/93a5a8ffb89245629058eac80dd9aaf9/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:43.998484 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164f892-af0d-427d-90c8-0c2f7d24125b admin admin] RBAC: Authorizing `identity:create_grant(project_id=38fd6f23aff249658b7b883c9bea9984, user_id=93a5a8ffb89245629058eac80dd9aaf9, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:44.005674 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164f892-af0d-427d-90c8-0c2f7d24125b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:44.016731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1022/2028] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:43 2022] PUT /identity/v3/projects/38fd6f23aff249658b7b883c9bea9984/users/93a5a8ffb89245629058eac80dd9aaf9/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:44.036856 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e437b9e-9def-4682-9ac6-bc108b573d9f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:44.038173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c50f4-0d4a-4d59-86a9-5cf60404b235 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:44.038561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c50f4-0d4a-4d59-86a9-5cf60404b235 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:44.038804 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c50f4-0d4a-4d59-86a9-5cf60404b235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:44.071683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-732c50f4-0d4a-4d59-86a9-5cf60404b235 None None] MFA Rules not processed for user `93a5a8ffb89245629058eac80dd9aaf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:44.073114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-732c50f4-0d4a-4d59-86a9-5cf60404b235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:44.114285 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f7b52-1a41-4cc7-8e77-cebf9471fc4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:44.114285 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f7b52-1a41-4cc7-8e77-cebf9471fc4b service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:44.114285 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f7b52-1a41-4cc7-8e77-cebf9471fc4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:44.114285 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f7b52-1a41-4cc7-8e77-cebf9471fc4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:44.173440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1023/2029] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:44.184875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c50f4-0d4a-4d59-86a9-5cf60404b235 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:44.186306 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-732c50f4-0d4a-4d59-86a9-5cf60404b235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:44.189956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f7b52-1a41-4cc7-8e77-cebf9471fc4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:44.192295 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f7b52-1a41-4cc7-8e77-cebf9471fc4b service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:44.202944 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1007/2030] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:44.249407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c50f4-0d4a-4d59-86a9-5cf60404b235 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:44.254063 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf81a18-3c17-4c0c-baed-1553167ea5f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:44.254757 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf81a18-3c17-4c0c-baed-1553167ea5f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:44.255044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf81a18-3c17-4c0c-baed-1553167ea5f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:44.255330 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf81a18-3c17-4c0c-baed-1553167ea5f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:44.257212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6cf81a18-3c17-4c0c-baed-1553167ea5f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:44.301282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:44.318701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf81a18-3c17-4c0c-baed-1553167ea5f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:44.320190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf81a18-3c17-4c0c-baed-1553167ea5f9 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:44.329253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1024/2031] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:44.354156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f7b52-1a41-4cc7-8e77-cebf9471fc4b service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:44.357031 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-aa1f7b52-1a41-4cc7-8e77-cebf9471fc4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:44.372244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f7b52-1a41-4cc7-8e77-cebf9471fc4b service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:44.383083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3910e-5307-40a8-81dd-07a2b9b8884d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:44.384034 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3910e-5307-40a8-81dd-07a2b9b8884d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:44.384335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3910e-5307-40a8-81dd-07a2b9b8884d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:44.384608 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3910e-5307-40a8-81dd-07a2b9b8884d admin admin] PATH_INFO: `/v3/projects/694e8fcffbe743b8a332c47c91cba2cf` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:44.386366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3910e-5307-40a8-81dd-07a2b9b8884d admin admin] RBAC: Authorizing `identity:delete_project(project_id=694e8fcffbe743b8a332c47c91cba2cf)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:44.389429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3910e-5307-40a8-81dd-07a2b9b8884d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:44.442211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:44.550841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-58a3910e-5307-40a8-81dd-07a2b9b8884d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '694e8fcffbe743b8a332c47c91cba2cf', 'request_id': 'req-58a3910e-5307-40a8-81dd-07a2b9b8884d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:44.558893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-58a3910e-5307-40a8-81dd-07a2b9b8884d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '694e8fcffbe743b8a332c47c91cba2cf', 'request_id': 'req-58a3910e-5307-40a8-81dd-07a2b9b8884d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:44.566794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-58a3910e-5307-40a8-81dd-07a2b9b8884d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '694e8fcffbe743b8a332c47c91cba2cf', 'request_id': 'req-58a3910e-5307-40a8-81dd-07a2b9b8884d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:44.693764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-58a3910e-5307-40a8-81dd-07a2b9b8884d admin admin] The token cache is being invalidate because project 694e8fcffbe743b8a332c47c91cba2cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:44.694426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-58a3910e-5307-40a8-81dd-07a2b9b8884d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:44.695254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-58a3910e-5307-40a8-81dd-07a2b9b8884d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:44.697558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1008/2032] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:44 2022] DELETE /identity/v3/projects/694e8fcffbe743b8a332c47c91cba2cf => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:44.705751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ca396-f656-4244-a0b3-8d3d25193436 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:44.705751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ca396-f656-4244-a0b3-8d3d25193436 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:44.705751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ca396-f656-4244-a0b3-8d3d25193436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:44.721726 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:44.722846 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5f9ca396-f656-4244-a0b3-8d3d25193436 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:44.724095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5f9ca396-f656-4244-a0b3-8d3d25193436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:44.784013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1025/2033] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:44.791699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08978212-1fe4-458e-bfa8-6049d3becc39 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:44.792001 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08978212-1fe4-458e-bfa8-6049d3becc39 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:44.792397 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08978212-1fe4-458e-bfa8-6049d3becc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:44.816821 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ca396-f656-4244-a0b3-8d3d25193436 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:44.818336 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5f9ca396-f656-4244-a0b3-8d3d25193436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:44.830774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-08978212-1fe4-458e-bfa8-6049d3becc39 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:44.834484 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-08978212-1fe4-458e-bfa8-6049d3becc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:44.834628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ca396-f656-4244-a0b3-8d3d25193436 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:44.841604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1009/2034] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:44.848349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f416ba-bad8-49bd-9350-48dbc28fa43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:44.849273 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f416ba-bad8-49bd-9350-48dbc28fa43b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:44.849565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f416ba-bad8-49bd-9350-48dbc28fa43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:44.849854 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f416ba-bad8-49bd-9350-48dbc28fa43b admin admin] PATH_INFO: `/v3/projects/8f87b19a30fe43bfbc9b5e2ff6dee915` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:44.851082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f416ba-bad8-49bd-9350-48dbc28fa43b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f87b19a30fe43bfbc9b5e2ff6dee915)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:44.851214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08978212-1fe4-458e-bfa8-6049d3becc39 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:44.853135 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f416ba-bad8-49bd-9350-48dbc28fa43b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:44.862390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08978212-1fe4-458e-bfa8-6049d3becc39 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:44.869460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343690e4-33da-4b1e-a92a-2923b010e9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:44.870144 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343690e4-33da-4b1e-a92a-2923b010e9ad admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:44.870436 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343690e4-33da-4b1e-a92a-2923b010e9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:44.870718 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343690e4-33da-4b1e-a92a-2923b010e9ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:44.871502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343690e4-33da-4b1e-a92a-2923b010e9ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:44.871813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343690e4-33da-4b1e-a92a-2923b010e9ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:44.873800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343690e4-33da-4b1e-a92a-2923b010e9ad admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:44.888580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1010/2035] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:42:44.894537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343690e4-33da-4b1e-a92a-2923b010e9ad admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:44.910040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343690e4-33da-4b1e-a92a-2923b010e9ad admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:44.914803 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6123b67-7737-41fb-ba0e-b14d4bdee79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:44.915442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6123b67-7737-41fb-ba0e-b14d4bdee79d admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:44.915733 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6123b67-7737-41fb-ba0e-b14d4bdee79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:44.916018 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6123b67-7737-41fb-ba0e-b14d4bdee79d admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:44.916711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6123b67-7737-41fb-ba0e-b14d4bdee79d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:44.918715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6123b67-7737-41fb-ba0e-b14d4bdee79d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:44.967535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70f416ba-bad8-49bd-9350-48dbc28fa43b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f87b19a30fe43bfbc9b5e2ff6dee915', 'request_id': 'req-70f416ba-bad8-49bd-9350-48dbc28fa43b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:44.975153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70f416ba-bad8-49bd-9350-48dbc28fa43b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f87b19a30fe43bfbc9b5e2ff6dee915', 'request_id': 'req-70f416ba-bad8-49bd-9350-48dbc28fa43b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:45.015602 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1011/2036] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:44 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:45.020224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6123b67-7737-41fb-ba0e-b14d4bdee79d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:45.040494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70f416ba-bad8-49bd-9350-48dbc28fa43b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f87b19a30fe43bfbc9b5e2ff6dee915', 'request_id': 'req-70f416ba-bad8-49bd-9350-48dbc28fa43b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:45.053380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70f416ba-bad8-49bd-9350-48dbc28fa43b admin admin] The token cache is being invalidate because project 8f87b19a30fe43bfbc9b5e2ff6dee915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:45.053687 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70f416ba-bad8-49bd-9350-48dbc28fa43b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:45.054373 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70f416ba-bad8-49bd-9350-48dbc28fa43b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:45.056318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1026/2037] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:44 2022] DELETE /identity/v3/projects/8f87b19a30fe43bfbc9b5e2ff6dee915 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:45.061221 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017256a2-f119-4e74-97ba-6f15b80946a4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.061440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017256a2-f119-4e74-97ba-6f15b80946a4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.061791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017256a2-f119-4e74-97ba-6f15b80946a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.098972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-017256a2-f119-4e74-97ba-6f15b80946a4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:45.100673 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-017256a2-f119-4e74-97ba-6f15b80946a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:45.115663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6123b67-7737-41fb-ba0e-b14d4bdee79d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:45.122400 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bc339-7cdd-4b40-9ccf-24b8d7e763f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:45.122848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bc339-7cdd-4b40-9ccf-24b8d7e763f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.122848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bc339-7cdd-4b40-9ccf-24b8d7e763f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.123025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1027/2038] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:45.123429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bc339-7cdd-4b40-9ccf-24b8d7e763f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.124253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bc339-7cdd-4b40-9ccf-24b8d7e763f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:45.124751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2436351-4ad2-4508-94cf-255403d1c97d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.125136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2436351-4ad2-4508-94cf-255403d1c97d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.125674 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2436351-4ad2-4508-94cf-255403d1c97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.126946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bc339-7cdd-4b40-9ccf-24b8d7e763f0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:45.153135 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d2436351-4ad2-4508-94cf-255403d1c97d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:45.154451 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d2436351-4ad2-4508-94cf-255403d1c97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:45.163768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1028/2039] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:45.168120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2436351-4ad2-4508-94cf-255403d1c97d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:45.178840 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2436351-4ad2-4508-94cf-255403d1c97d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:45.183970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63868d37-9dac-41b2-a963-fc52fba1f070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:45.184626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63868d37-9dac-41b2-a963-fc52fba1f070 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.184912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63868d37-9dac-41b2-a963-fc52fba1f070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.185220 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63868d37-9dac-41b2-a963-fc52fba1f070 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.185950 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63868d37-9dac-41b2-a963-fc52fba1f070 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:45.186256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63868d37-9dac-41b2-a963-fc52fba1f070 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:45.187879 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63868d37-9dac-41b2-a963-fc52fba1f070 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:45.199664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1029/2040] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:42:45.204185 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63868d37-9dac-41b2-a963-fc52fba1f070 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:45.230224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63868d37-9dac-41b2-a963-fc52fba1f070 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:45.230879 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1012/2041] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:45 2022] POST /identity/v3/users => generated 291 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:45.235430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bc339-7cdd-4b40-9ccf-24b8d7e763f0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:45.237262 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493d713-6d19-42ae-ba15-93dffaa28a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:45.237423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c22bc339-7cdd-4b40-9ccf-24b8d7e763f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:45.238059 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493d713-6d19-42ae-ba15-93dffaa28a62 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.238651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493d713-6d19-42ae-ba15-93dffaa28a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.239045 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493d713-6d19-42ae-ba15-93dffaa28a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.240019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493d713-6d19-42ae-ba15-93dffaa28a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:45.242007 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493d713-6d19-42ae-ba15-93dffaa28a62 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:45.250547 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bc339-7cdd-4b40-9ccf-24b8d7e763f0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:45.256161 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ecec8-0a80-4085-a484-e5d294cd4d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:45.256854 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ecec8-0a80-4085-a484-e5d294cd4d6c admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.257232 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ecec8-0a80-4085-a484-e5d294cd4d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.257440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ecec8-0a80-4085-a484-e5d294cd4d6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.258221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ecec8-0a80-4085-a484-e5d294cd4d6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:45.259831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ecec8-0a80-4085-a484-e5d294cd4d6c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:45.270685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1013/2042] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:45.274421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ecec8-0a80-4085-a484-e5d294cd4d6c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:45.284848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ecec8-0a80-4085-a484-e5d294cd4d6c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:45.289553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423f638-ac87-4bcc-8c4f-47a886551861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:45.290206 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423f638-ac87-4bcc-8c4f-47a886551861 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.290492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423f638-ac87-4bcc-8c4f-47a886551861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.290867 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423f638-ac87-4bcc-8c4f-47a886551861 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.291537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423f638-ac87-4bcc-8c4f-47a886551861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:45.293156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423f638-ac87-4bcc-8c4f-47a886551861 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:45.303913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1014/2043] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:45.309649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423f638-ac87-4bcc-8c4f-47a886551861 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:45.317292 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423f638-ac87-4bcc-8c4f-47a886551861 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:45.321834 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c2d32-8d21-4a6e-b376-e77c49b4f2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:45.322706 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c2d32-8d21-4a6e-b376-e77c49b4f2b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.322991 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c2d32-8d21-4a6e-b376-e77c49b4f2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.323275 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c2d32-8d21-4a6e-b376-e77c49b4f2b9 admin admin] PATH_INFO: `/v3/projects/3801151d59fa43d2afcb16d48501ead9/users/670584e952fc4947ab7e6effc3f8f077/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.334887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1030/2044] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:45 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:45.340118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c2d32-8d21-4a6e-b376-e77c49b4f2b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3801151d59fa43d2afcb16d48501ead9, user_id=670584e952fc4947ab7e6effc3f8f077, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:45.340979 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c2d32-8d21-4a6e-b376-e77c49b4f2b9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:45.341387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493d713-6d19-42ae-ba15-93dffaa28a62 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:45.402943 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493d713-6d19-42ae-ba15-93dffaa28a62 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:45.408841 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097203d4-f628-4563-86b4-71be636e8de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:45.409545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097203d4-f628-4563-86b4-71be636e8de8 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.409861 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097203d4-f628-4563-86b4-71be636e8de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.410140 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097203d4-f628-4563-86b4-71be636e8de8 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.410858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097203d4-f628-4563-86b4-71be636e8de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:45.412766 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097203d4-f628-4563-86b4-71be636e8de8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:45.430114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1015/2045] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:45 2022] PUT /identity/v3/projects/3801151d59fa43d2afcb16d48501ead9/users/670584e952fc4947ab7e6effc3f8f077/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:45.440528 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603e5b9-7486-4d8e-9f1e-46a1fdda2a5a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.440836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603e5b9-7486-4d8e-9f1e-46a1fdda2a5a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.441158 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603e5b9-7486-4d8e-9f1e-46a1fdda2a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.467729 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f603e5b9-7486-4d8e-9f1e-46a1fdda2a5a None None] MFA Rules not processed for user `670584e952fc4947ab7e6effc3f8f077`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:45.468959 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f603e5b9-7486-4d8e-9f1e-46a1fdda2a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:45.498849 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1031/2046] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:45 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:45.502896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097203d4-f628-4563-86b4-71be636e8de8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:45.512377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097203d4-f628-4563-86b4-71be636e8de8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:45.535261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1016/2047] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:45.539710 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603e5b9-7486-4d8e-9f1e-46a1fdda2a5a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:45.572092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130b22f-91ca-4edc-b64b-675966de6d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:45.572966 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130b22f-91ca-4edc-b64b-675966de6d51 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.573295 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130b22f-91ca-4edc-b64b-675966de6d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.573712 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130b22f-91ca-4edc-b64b-675966de6d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.574315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130b22f-91ca-4edc-b64b-675966de6d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:45.575952 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130b22f-91ca-4edc-b64b-675966de6d51 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:45.587600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1032/2048] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:45.591421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130b22f-91ca-4edc-b64b-675966de6d51 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:45.601137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603e5b9-7486-4d8e-9f1e-46a1fdda2a5a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:45.601475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130b22f-91ca-4edc-b64b-675966de6d51 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:45.605770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c7e10-623c-4dc5-9808-28b305ed694f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:45.606017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e2db5-a41b-47ad-aaf5-ea0c1e1d7c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:45.606442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c7e10-623c-4dc5-9808-28b305ed694f admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.606660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e2db5-a41b-47ad-aaf5-ea0c1e1d7c0f admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.606767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c7e10-623c-4dc5-9808-28b305ed694f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.606952 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e2db5-a41b-47ad-aaf5-ea0c1e1d7c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.607066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c7e10-623c-4dc5-9808-28b305ed694f admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.607230 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e2db5-a41b-47ad-aaf5-ea0c1e1d7c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.607683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c7e10-623c-4dc5-9808-28b305ed694f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:45.607943 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e2db5-a41b-47ad-aaf5-ea0c1e1d7c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:45.609562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e2db5-a41b-47ad-aaf5-ea0c1e1d7c0f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:45.609804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c7e10-623c-4dc5-9808-28b305ed694f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:45.619393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1033/2049] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:45.623295 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e2db5-a41b-47ad-aaf5-ea0c1e1d7c0f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:45.626352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:45.633445 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e2db5-a41b-47ad-aaf5-ea0c1e1d7c0f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:45.638208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb0dc3-c7b5-4a00-9612-feac87f8c0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:45.639075 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb0dc3-c7b5-4a00-9612-feac87f8c0af admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.639362 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb0dc3-c7b5-4a00-9612-feac87f8c0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.639644 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb0dc3-c7b5-4a00-9612-feac87f8c0af admin admin] PATH_INFO: `/v3/projects/e0860b3d072246d49cd43a900351d571/users/8b983c0a2a124ac3aa3403e0777612f7/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.654739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb0dc3-c7b5-4a00-9612-feac87f8c0af admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0860b3d072246d49cd43a900351d571, user_id=8b983c0a2a124ac3aa3403e0777612f7, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:45.657676 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb0dc3-c7b5-4a00-9612-feac87f8c0af admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:45.671600 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1017/2050] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:45 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:45.676019 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c7e10-623c-4dc5-9808-28b305ed694f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:45.743021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c7e10-623c-4dc5-9808-28b305ed694f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:45.747614 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959a5ab-aa86-44a2-b251-2e2592b3cee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:45.748312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959a5ab-aa86-44a2-b251-2e2592b3cee4 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.748608 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959a5ab-aa86-44a2-b251-2e2592b3cee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.748884 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959a5ab-aa86-44a2-b251-2e2592b3cee4 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.749614 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7959a5ab-aa86-44a2-b251-2e2592b3cee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:45.752296 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7959a5ab-aa86-44a2-b251-2e2592b3cee4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:45.756976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1034/2051] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:45 2022] PUT /identity/v3/projects/e0860b3d072246d49cd43a900351d571/users/8b983c0a2a124ac3aa3403e0777612f7/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:45.762553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:45.768532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130a609-5bf8-4a09-9c59-5b1aa49ac28c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.768532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130a609-5bf8-4a09-9c59-5b1aa49ac28c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.768532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130a609-5bf8-4a09-9c59-5b1aa49ac28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.783314 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e130a609-5bf8-4a09-9c59-5b1aa49ac28c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:45.784517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e130a609-5bf8-4a09-9c59-5b1aa49ac28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:45.841991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1035/2052] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:45.851359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130a609-5bf8-4a09-9c59-5b1aa49ac28c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:45.852652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e130a609-5bf8-4a09-9c59-5b1aa49ac28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:45.894790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1018/2053] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:45 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:45.899376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959a5ab-aa86-44a2-b251-2e2592b3cee4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:45.910308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959a5ab-aa86-44a2-b251-2e2592b3cee4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:45.912004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130a609-5bf8-4a09-9c59-5b1aa49ac28c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:45.915299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d21efb-12fb-455d-8b0c-b9b440a5301a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:45.915443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a07027-b392-450b-a052-0fad1d510b89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:45.916022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a07027-b392-450b-a052-0fad1d510b89 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.916197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d21efb-12fb-455d-8b0c-b9b440a5301a admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.916321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a07027-b392-450b-a052-0fad1d510b89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.916602 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d21efb-12fb-455d-8b0c-b9b440a5301a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.916711 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a07027-b392-450b-a052-0fad1d510b89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.917102 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d21efb-12fb-455d-8b0c-b9b440a5301a admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.917856 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d21efb-12fb-455d-8b0c-b9b440a5301a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:45.919801 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d21efb-12fb-455d-8b0c-b9b440a5301a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:45.927270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a07027-b392-450b-a052-0fad1d510b89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:45.929200 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a07027-b392-450b-a052-0fad1d510b89 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:45.935000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1019/2054] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:45.936468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d21efb-12fb-455d-8b0c-b9b440a5301a admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:45.941151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1036/2055] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:45.953176 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d21efb-12fb-455d-8b0c-b9b440a5301a admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:45.953176 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273e0bb-4ca4-4f56-98bd-c42907ff4f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:45.954160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273e0bb-4ca4-4f56-98bd-c42907ff4f08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:45.954934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273e0bb-4ca4-4f56-98bd-c42907ff4f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:45.955235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273e0bb-4ca4-4f56-98bd-c42907ff4f08 admin admin] PATH_INFO: `/v3/projects/8fe77a0150984523b972df85f7a097e0/users/f79e100580fe4b8783191fa40d364a44/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:45.972847 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273e0bb-4ca4-4f56-98bd-c42907ff4f08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe77a0150984523b972df85f7a097e0, user_id=f79e100580fe4b8783191fa40d364a44, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:45.975588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273e0bb-4ca4-4f56-98bd-c42907ff4f08 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:46.029832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:46.076638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1020/2056] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:45 2022] PUT /identity/v3/projects/8fe77a0150984523b972df85f7a097e0/users/f79e100580fe4b8783191fa40d364a44/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:46.080875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a07027-b392-450b-a052-0fad1d510b89 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:46.091140 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a07027-b392-450b-a052-0fad1d510b89 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:46.145932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:46.155814 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53304e44-46f9-45de-9106-63e2eb11ad52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:46.156521 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53304e44-46f9-45de-9106-63e2eb11ad52 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:46.156865 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53304e44-46f9-45de-9106-63e2eb11ad52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:46.157351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53304e44-46f9-45de-9106-63e2eb11ad52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:46.158176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53304e44-46f9-45de-9106-63e2eb11ad52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:46.160092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53304e44-46f9-45de-9106-63e2eb11ad52 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:46.174491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1037/2057] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:46.175788 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273e0bb-4ca4-4f56-98bd-c42907ff4f08 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:46.186462 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273e0bb-4ca4-4f56-98bd-c42907ff4f08 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:46.192401 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ceb972-758b-4b6c-88d7-2f30de18ca0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:46.193325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ceb972-758b-4b6c-88d7-2f30de18ca0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:46.193615 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ceb972-758b-4b6c-88d7-2f30de18ca0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:46.193896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ceb972-758b-4b6c-88d7-2f30de18ca0b admin admin] PATH_INFO: `/v3/projects/8fe77a0150984523b972df85f7a097e0/users/f79e100580fe4b8783191fa40d364a44/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:46.195993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ceb972-758b-4b6c-88d7-2f30de18ca0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe77a0150984523b972df85f7a097e0, user_id=f79e100580fe4b8783191fa40d364a44, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:46.198744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ceb972-758b-4b6c-88d7-2f30de18ca0b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:46.210174 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1021/2058] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:46 2022] PUT /identity/v3/projects/8fe77a0150984523b972df85f7a097e0/users/f79e100580fe4b8783191fa40d364a44/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:46.221569 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913513b4-d1bd-4f12-bec5-1641386a2a9d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:46.221951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913513b4-d1bd-4f12-bec5-1641386a2a9d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:46.222297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913513b4-d1bd-4f12-bec5-1641386a2a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:46.286609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-913513b4-d1bd-4f12-bec5-1641386a2a9d None None] MFA Rules not processed for user `f79e100580fe4b8783191fa40d364a44`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:46.288082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-913513b4-d1bd-4f12-bec5-1641386a2a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:46.383606 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1038/2059] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:46.668496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ceb972-758b-4b6c-88d7-2f30de18ca0b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:46.670791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-43ceb972-758b-4b6c-88d7-2f30de18ca0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:46.764101 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ceb972-758b-4b6c-88d7-2f30de18ca0b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:46.785333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d96d45-c912-4d9f-86ac-a2f45ae2b207 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:46.786017 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d96d45-c912-4d9f-86ac-a2f45ae2b207 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:46.786310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d96d45-c912-4d9f-86ac-a2f45ae2b207 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:46.786592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d96d45-c912-4d9f-86ac-a2f45ae2b207 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:46.799705 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d96d45-c912-4d9f-86ac-a2f45ae2b207 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:46.801511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d96d45-c912-4d9f-86ac-a2f45ae2b207 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:46.811489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1022/2060] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:47.025170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:47.415142 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913513b4-d1bd-4f12-bec5-1641386a2a9d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:47.477025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913513b4-d1bd-4f12-bec5-1641386a2a9d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:47.481584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e081a-9799-4a9e-b2ab-c71907cde135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:47.482488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e081a-9799-4a9e-b2ab-c71907cde135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:47.482770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e081a-9799-4a9e-b2ab-c71907cde135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:47.483044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e081a-9799-4a9e-b2ab-c71907cde135 admin admin] PATH_INFO: `/v3/users/670584e952fc4947ab7e6effc3f8f077` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:47.484237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e081a-9799-4a9e-b2ab-c71907cde135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=670584e952fc4947ab7e6effc3f8f077)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:47.486767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e081a-9799-4a9e-b2ab-c71907cde135 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:47.727434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:47.825849 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3f1e081a-9799-4a9e-b2ab-c71907cde135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '670584e952fc4947ab7e6effc3f8f077'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:47.839398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3f1e081a-9799-4a9e-b2ab-c71907cde135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '670584e952fc4947ab7e6effc3f8f077'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:47.933076 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3f1e081a-9799-4a9e-b2ab-c71907cde135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '670584e952fc4947ab7e6effc3f8f077'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:47.933743 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3f1e081a-9799-4a9e-b2ab-c71907cde135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '670584e952fc4947ab7e6effc3f8f077'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:47.941097 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3f1e081a-9799-4a9e-b2ab-c71907cde135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '670584e952fc4947ab7e6effc3f8f077'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:47.947031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1039/2061] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:47 2022] DELETE /identity/v3/users/670584e952fc4947ab7e6effc3f8f077 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:47.951540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4af80c-f53e-4d6c-948b-f00257688db6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:47.952020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4af80c-f53e-4d6c-948b-f00257688db6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:47.952219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4af80c-f53e-4d6c-948b-f00257688db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:47.970681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8b4af80c-f53e-4d6c-948b-f00257688db6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:47.971878 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8b4af80c-f53e-4d6c-948b-f00257688db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:48.046820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1023/2062] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:48.058735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e081a-9799-4a9e-b2ab-c71907cde135 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:48.060175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3f1e081a-9799-4a9e-b2ab-c71907cde135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:48.127174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e081a-9799-4a9e-b2ab-c71907cde135 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:48.130932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628f333-d681-4f91-bdc5-d6828965ef03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:48.131580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628f333-d681-4f91-bdc5-d6828965ef03 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:48.131866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628f333-d681-4f91-bdc5-d6828965ef03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:48.132139 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628f333-d681-4f91-bdc5-d6828965ef03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:48.141544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628f333-d681-4f91-bdc5-d6828965ef03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:48.143096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628f333-d681-4f91-bdc5-d6828965ef03 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:48.153148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1040/2063] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:48.556342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:48.830014 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4af80c-f53e-4d6c-948b-f00257688db6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:48.831455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8b4af80c-f53e-4d6c-948b-f00257688db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:48.847687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4af80c-f53e-4d6c-948b-f00257688db6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:48.859458 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:48.862069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:48.862069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:48.863032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b admin admin] PATH_INFO: `/v3/projects/3801151d59fa43d2afcb16d48501ead9` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:48.864408 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3801151d59fa43d2afcb16d48501ead9)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:48.866578 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:49.039815 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3801151d59fa43d2afcb16d48501ead9', 'request_id': 'req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:49.047964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3801151d59fa43d2afcb16d48501ead9', 'request_id': 'req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:49.055451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3801151d59fa43d2afcb16d48501ead9', 'request_id': 'req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:49.140010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b admin admin] The token cache is being invalidate because project 3801151d59fa43d2afcb16d48501ead9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:49.141823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:49.145403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:49.147218 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1024/2064] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:48 2022] DELETE /identity/v3/projects/3801151d59fa43d2afcb16d48501ead9 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:49.151129 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628f333-d681-4f91-bdc5-d6828965ef03 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:49.163146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628f333-d681-4f91-bdc5-d6828965ef03 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:49.233571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d256ef-0a29-4ab7-96b8-87a32c073859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:49.234464 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d256ef-0a29-4ab7-96b8-87a32c073859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:49.234794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d256ef-0a29-4ab7-96b8-87a32c073859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:49.235070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d256ef-0a29-4ab7-96b8-87a32c073859 admin admin] PATH_INFO: `/v3/users/f79e100580fe4b8783191fa40d364a44` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:49.236300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d256ef-0a29-4ab7-96b8-87a32c073859 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f79e100580fe4b8783191fa40d364a44)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:49.238276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d256ef-0a29-4ab7-96b8-87a32c073859 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:49.430358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b9d256ef-0a29-4ab7-96b8-87a32c073859 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f79e100580fe4b8783191fa40d364a44'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:49.447118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b9d256ef-0a29-4ab7-96b8-87a32c073859 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f79e100580fe4b8783191fa40d364a44'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:49.544500 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b9d256ef-0a29-4ab7-96b8-87a32c073859 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f79e100580fe4b8783191fa40d364a44'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:49.545238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b9d256ef-0a29-4ab7-96b8-87a32c073859 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f79e100580fe4b8783191fa40d364a44'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:49.554379 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b9d256ef-0a29-4ab7-96b8-87a32c073859 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f79e100580fe4b8783191fa40d364a44'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:49.564464 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1041/2065] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:49 2022] DELETE /identity/v3/users/f79e100580fe4b8783191fa40d364a44 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:49.681692 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:50.015564 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:50.018324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:50.033755 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abc08d-d0e0-48d6-aabd-0180733bfa8b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:50.130101 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff103fe-5b11-478b-81db-837ba7e811c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:50.132076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff103fe-5b11-478b-81db-837ba7e811c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:50.132512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff103fe-5b11-478b-81db-837ba7e811c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:50.132976 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff103fe-5b11-478b-81db-837ba7e811c1 admin admin] PATH_INFO: `/v3/projects/8fe77a0150984523b972df85f7a097e0` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:50.138480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff103fe-5b11-478b-81db-837ba7e811c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fe77a0150984523b972df85f7a097e0)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:50.138480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff103fe-5b11-478b-81db-837ba7e811c1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:50.295788 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-aff103fe-5b11-478b-81db-837ba7e811c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fe77a0150984523b972df85f7a097e0', 'request_id': 'req-aff103fe-5b11-478b-81db-837ba7e811c1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:50.304282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-aff103fe-5b11-478b-81db-837ba7e811c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fe77a0150984523b972df85f7a097e0', 'request_id': 'req-aff103fe-5b11-478b-81db-837ba7e811c1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:50.315997 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-aff103fe-5b11-478b-81db-837ba7e811c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fe77a0150984523b972df85f7a097e0', 'request_id': 'req-aff103fe-5b11-478b-81db-837ba7e811c1'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:50.460943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-aff103fe-5b11-478b-81db-837ba7e811c1 admin admin] The token cache is being invalidate because project 8fe77a0150984523b972df85f7a097e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:50.463008 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-aff103fe-5b11-478b-81db-837ba7e811c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:50.465788 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-aff103fe-5b11-478b-81db-837ba7e811c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:50.471722 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1025/2066] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:50 2022] DELETE /identity/v3/projects/8fe77a0150984523b972df85f7a097e0 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:50.478963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce603a16-8b61-4f06-b9e3-f828ecc88591 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:50.479921 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce603a16-8b61-4f06-b9e3-f828ecc88591 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:50.480544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce603a16-8b61-4f06-b9e3-f828ecc88591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:50.513668 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ce603a16-8b61-4f06-b9e3-f828ecc88591 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:50.515300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ce603a16-8b61-4f06-b9e3-f828ecc88591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:50.636329 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1042/2067] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:50.646731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b08dc0-c793-4408-93d4-bf65a96575c4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:50.647148 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b08dc0-c793-4408-93d4-bf65a96575c4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:50.647663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b08dc0-c793-4408-93d4-bf65a96575c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:50.669700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-20b08dc0-c793-4408-93d4-bf65a96575c4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:50.671283 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-20b08dc0-c793-4408-93d4-bf65a96575c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:50.680317 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1026/2068] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:50.684920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce603a16-8b61-4f06-b9e3-f828ecc88591 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:50.694994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce603a16-8b61-4f06-b9e3-f828ecc88591 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:50.700128 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810979bf-fb81-4a0f-b576-132fc45b2ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:50.701592 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810979bf-fb81-4a0f-b576-132fc45b2ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:50.702916 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810979bf-fb81-4a0f-b576-132fc45b2ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:50.704013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810979bf-fb81-4a0f-b576-132fc45b2ce6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:50.705108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810979bf-fb81-4a0f-b576-132fc45b2ce6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:50.705574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810979bf-fb81-4a0f-b576-132fc45b2ce6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:50.708335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810979bf-fb81-4a0f-b576-132fc45b2ce6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:50.725147 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1043/2069] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:42:50.729977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b08dc0-c793-4408-93d4-bf65a96575c4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:50.739612 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b08dc0-c793-4408-93d4-bf65a96575c4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:50.750092 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bdcf7-13ba-409f-862a-f3da9885d617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:50.750985 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bdcf7-13ba-409f-862a-f3da9885d617 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:50.751279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bdcf7-13ba-409f-862a-f3da9885d617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:50.751561 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bdcf7-13ba-409f-862a-f3da9885d617 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:50.752285 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bdcf7-13ba-409f-862a-f3da9885d617 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:50.754279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bdcf7-13ba-409f-862a-f3da9885d617 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:50.813509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810979bf-fb81-4a0f-b576-132fc45b2ce6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:50.817027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-810979bf-fb81-4a0f-b576-132fc45b2ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:50.832485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810979bf-fb81-4a0f-b576-132fc45b2ce6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:50.838370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203a6f2-0535-4f44-b75d-2822b8f486da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:50.839271 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203a6f2-0535-4f44-b75d-2822b8f486da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:50.839558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203a6f2-0535-4f44-b75d-2822b8f486da admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:50.839972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203a6f2-0535-4f44-b75d-2822b8f486da admin admin] PATH_INFO: `/v3/users/5a9411b008b34be6b948ede03e1acbfc` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:50.841441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203a6f2-0535-4f44-b75d-2822b8f486da admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a9411b008b34be6b948ede03e1acbfc)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:50.843373 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203a6f2-0535-4f44-b75d-2822b8f486da admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:50.862176 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1027/2070] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:50 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:50.866360 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bdcf7-13ba-409f-862a-f3da9885d617 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:50.959270 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bdcf7-13ba-409f-862a-f3da9885d617 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:50.967491 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f76d2-addc-4259-898e-7581f981a55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:50.968434 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f76d2-addc-4259-898e-7581f981a55c admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:50.968828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f76d2-addc-4259-898e-7581f981a55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:50.969989 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f76d2-addc-4259-898e-7581f981a55c admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:50.971315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f76d2-addc-4259-898e-7581f981a55c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:50.974276 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f76d2-addc-4259-898e-7581f981a55c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:51.014056 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:51.085107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1028/2071] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:50 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:51.088767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f76d2-addc-4259-898e-7581f981a55c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:51.092333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9e7f76d2-addc-4259-898e-7581f981a55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:51.104805 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6203a6f2-0535-4f44-b75d-2822b8f486da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a9411b008b34be6b948ede03e1acbfc'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:51.105701 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f76d2-addc-4259-898e-7581f981a55c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:51.111859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac6ea0-19bb-4c73-be6d-4e2e76cd5f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:51.112934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac6ea0-19bb-4c73-be6d-4e2e76cd5f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:51.113396 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac6ea0-19bb-4c73-be6d-4e2e76cd5f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:51.113825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac6ea0-19bb-4c73-be6d-4e2e76cd5f79 admin admin] PATH_INFO: `/v3/users/27ed4d43434c448a9f890d2965a606c2` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:51.115496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac6ea0-19bb-4c73-be6d-4e2e76cd5f79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27ed4d43434c448a9f890d2965a606c2)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:51.117957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac6ea0-19bb-4c73-be6d-4e2e76cd5f79 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:51.119530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6203a6f2-0535-4f44-b75d-2822b8f486da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a9411b008b34be6b948ede03e1acbfc'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:51.178916 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6203a6f2-0535-4f44-b75d-2822b8f486da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a9411b008b34be6b948ede03e1acbfc'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:51.179552 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6203a6f2-0535-4f44-b75d-2822b8f486da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a9411b008b34be6b948ede03e1acbfc'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:51.193319 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6203a6f2-0535-4f44-b75d-2822b8f486da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a9411b008b34be6b948ede03e1acbfc'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:51.202297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1044/2072] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:50 2022] DELETE /identity/v3/users/5a9411b008b34be6b948ede03e1acbfc => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:51.204618 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203a6f2-0535-4f44-b75d-2822b8f486da admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:51.207253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6203a6f2-0535-4f44-b75d-2822b8f486da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:51.313741 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203a6f2-0535-4f44-b75d-2822b8f486da admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:51.319704 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a25dfa-61bf-4bd0-9b6e-2b4042629579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:51.320648 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a25dfa-61bf-4bd0-9b6e-2b4042629579 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:51.320997 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a25dfa-61bf-4bd0-9b6e-2b4042629579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:51.321399 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a25dfa-61bf-4bd0-9b6e-2b4042629579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:51.322254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a25dfa-61bf-4bd0-9b6e-2b4042629579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:51.324326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a25dfa-61bf-4bd0-9b6e-2b4042629579 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:51.338750 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1045/2073] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:51.340337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc05ef-335e-4a1f-856d-2cff6eb05ad2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:51.340730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc05ef-335e-4a1f-856d-2cff6eb05ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:51.341136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc05ef-335e-4a1f-856d-2cff6eb05ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:51.372062 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-26bc05ef-335e-4a1f-856d-2cff6eb05ad2 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:51.377587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-26bc05ef-335e-4a1f-856d-2cff6eb05ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:51.381295 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-95ac6ea0-19bb-4c73-be6d-4e2e76cd5f79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27ed4d43434c448a9f890d2965a606c2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:51.382675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-95ac6ea0-19bb-4c73-be6d-4e2e76cd5f79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27ed4d43434c448a9f890d2965a606c2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:51.383578 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-95ac6ea0-19bb-4c73-be6d-4e2e76cd5f79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27ed4d43434c448a9f890d2965a606c2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:51.394874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-95ac6ea0-19bb-4c73-be6d-4e2e76cd5f79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27ed4d43434c448a9f890d2965a606c2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:51.399296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1046/2074] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:51.403537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc05ef-335e-4a1f-856d-2cff6eb05ad2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:51.405091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-26bc05ef-335e-4a1f-856d-2cff6eb05ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:51.420291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-95ac6ea0-19bb-4c73-be6d-4e2e76cd5f79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27ed4d43434c448a9f890d2965a606c2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:51.420508 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc05ef-335e-4a1f-856d-2cff6eb05ad2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:51.431090 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129e328-c4a4-40f0-9727-0eb33af8a0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:51.433253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129e328-c4a4-40f0-9727-0eb33af8a0d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:51.433253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129e328-c4a4-40f0-9727-0eb33af8a0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:51.433253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129e328-c4a4-40f0-9727-0eb33af8a0d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:51.433776 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129e328-c4a4-40f0-9727-0eb33af8a0d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:51.435583 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129e328-c4a4-40f0-9727-0eb33af8a0d1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:51.465634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1047/2075] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:51.466872 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129e328-c4a4-40f0-9727-0eb33af8a0d1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:51.468847 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7129e328-c4a4-40f0-9727-0eb33af8a0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:51.533844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1029/2076] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:51 2022] DELETE /identity/v3/users/27ed4d43434c448a9f890d2965a606c2 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:51.534790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac6ea0-19bb-4c73-be6d-4e2e76cd5f79 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:51.586368 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129e328-c4a4-40f0-9727-0eb33af8a0d1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:51.645011 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac6ea0-19bb-4c73-be6d-4e2e76cd5f79 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:51.654527 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4b8fc-d81b-4173-a09e-be7712a718d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:51.656135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4b8fc-d81b-4173-a09e-be7712a718d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:51.659613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4b8fc-d81b-4173-a09e-be7712a718d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:51.659613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4b8fc-d81b-4173-a09e-be7712a718d5 admin admin] PATH_INFO: `/v3/projects/b9cf3418aae24d11b805c3871ac18dee/users/ab17209304f248d4ac482b59ce537187/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:51.680915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea59a35-5a9e-4511-a43f-ef0e64c4f5fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:51.681747 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4b8fc-d81b-4173-a09e-be7712a718d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9cf3418aae24d11b805c3871ac18dee, user_id=ab17209304f248d4ac482b59ce537187, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:51.681865 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea59a35-5a9e-4511-a43f-ef0e64c4f5fe service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:51.681955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea59a35-5a9e-4511-a43f-ef0e64c4f5fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:51.682754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea59a35-5a9e-4511-a43f-ef0e64c4f5fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:51.685494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4b8fc-d81b-4173-a09e-be7712a718d5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:51.711255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea59a35-5a9e-4511-a43f-ef0e64c4f5fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:51.712453 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea59a35-5a9e-4511-a43f-ef0e64c4f5fe service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:51.741366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1048/2077] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:51.742766 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b5934-04ce-406a-b139-0744dfa67aad None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:51.745258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b5934-04ce-406a-b139-0744dfa67aad None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:51.745815 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b5934-04ce-406a-b139-0744dfa67aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:51.765089 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fb9b5934-04ce-406a-b139-0744dfa67aad None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:51.767304 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fb9b5934-04ce-406a-b139-0744dfa67aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:51.785434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1049/2078] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:51.801006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b5934-04ce-406a-b139-0744dfa67aad None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:51.801346 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1030/2079] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:51 2022] PUT /identity/v3/projects/b9cf3418aae24d11b805c3871ac18dee/users/ab17209304f248d4ac482b59ce537187/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:51.803600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fb9b5934-04ce-406a-b139-0744dfa67aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:51.811991 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fa6d4-f96e-4ef5-8868-f5e339a3faa5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:51.812416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fa6d4-f96e-4ef5-8868-f5e339a3faa5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:51.812856 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fa6d4-f96e-4ef5-8868-f5e339a3faa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:51.818524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b5934-04ce-406a-b139-0744dfa67aad None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:51.875974 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8d1fa6d4-f96e-4ef5-8868-f5e339a3faa5 None None] MFA Rules not processed for user `ab17209304f248d4ac482b59ce537187`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:51.881486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8d1fa6d4-f96e-4ef5-8868-f5e339a3faa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:51.936779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386a5c5-65d4-42df-8d11-a4d5f40aea4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:51.938869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386a5c5-65d4-42df-8d11-a4d5f40aea4b service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:51.939169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386a5c5-65d4-42df-8d11-a4d5f40aea4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:51.939463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386a5c5-65d4-42df-8d11-a4d5f40aea4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:52.017681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1031/2080] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:42:52.042938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fa6d4-f96e-4ef5-8868-f5e339a3faa5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:52.042938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8d1fa6d4-f96e-4ef5-8868-f5e339a3faa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:52.058064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386a5c5-65d4-42df-8d11-a4d5f40aea4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:52.061538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386a5c5-65d4-42df-8d11-a4d5f40aea4b service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:52.077249 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1050/2081] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:52.080625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:52.128091 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fa6d4-f96e-4ef5-8868-f5e339a3faa5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:52.134890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd562b1-d542-44aa-8dd0-3d4c2a209647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:52.135784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd562b1-d542-44aa-8dd0-3d4c2a209647 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:52.136270 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd562b1-d542-44aa-8dd0-3d4c2a209647 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:52.137046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd562b1-d542-44aa-8dd0-3d4c2a209647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:52.151339 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd562b1-d542-44aa-8dd0-3d4c2a209647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:52.152881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd562b1-d542-44aa-8dd0-3d4c2a209647 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:52.181496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1032/2082] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:52.200525 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386a5c5-65d4-42df-8d11-a4d5f40aea4b service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:52.211959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386a5c5-65d4-42df-8d11-a4d5f40aea4b service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:52.224163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9800759-4677-48c5-bb28-dae8e8f4a910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:52.226235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9800759-4677-48c5-bb28-dae8e8f4a910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:52.227289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9800759-4677-48c5-bb28-dae8e8f4a910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:52.227739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9800759-4677-48c5-bb28-dae8e8f4a910 admin admin] PATH_INFO: `/v3/users/ab17209304f248d4ac482b59ce537187` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:52.237564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9800759-4677-48c5-bb28-dae8e8f4a910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab17209304f248d4ac482b59ce537187)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:52.240396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9800759-4677-48c5-bb28-dae8e8f4a910 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:52.491913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f9800759-4677-48c5-bb28-dae8e8f4a910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab17209304f248d4ac482b59ce537187'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:52.508135 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f9800759-4677-48c5-bb28-dae8e8f4a910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab17209304f248d4ac482b59ce537187'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:52.619413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f9800759-4677-48c5-bb28-dae8e8f4a910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab17209304f248d4ac482b59ce537187'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:52.620136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f9800759-4677-48c5-bb28-dae8e8f4a910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab17209304f248d4ac482b59ce537187'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:52.635162 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f9800759-4677-48c5-bb28-dae8e8f4a910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab17209304f248d4ac482b59ce537187'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:52.649632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1051/2083] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:52 2022] DELETE /identity/v3/users/ab17209304f248d4ac482b59ce537187 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:52.654098 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12af58b-970a-468c-bc7c-f02c561d2576 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:52.658853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12af58b-970a-468c-bc7c-f02c561d2576 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:52.658853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12af58b-970a-468c-bc7c-f02c561d2576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:52.670380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9800759-4677-48c5-bb28-dae8e8f4a910 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:52.673540 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f9800759-4677-48c5-bb28-dae8e8f4a910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:52.690939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d12af58b-970a-468c-bc7c-f02c561d2576 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:52.692419 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d12af58b-970a-468c-bc7c-f02c561d2576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:52.801731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9800759-4677-48c5-bb28-dae8e8f4a910 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:52.804704 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d0cd2-ffcb-4bd7-99e8-18d5c9fb8767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:52.805966 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d0cd2-ffcb-4bd7-99e8-18d5c9fb8767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:52.805966 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d0cd2-ffcb-4bd7-99e8-18d5c9fb8767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:52.806121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d0cd2-ffcb-4bd7-99e8-18d5c9fb8767 admin admin] PATH_INFO: `/v3/projects/ffba61b8eb664af5a7f968f90f9513bd` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:52.807348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d0cd2-ffcb-4bd7-99e8-18d5c9fb8767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffba61b8eb664af5a7f968f90f9513bd)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:52.809324 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d0cd2-ffcb-4bd7-99e8-18d5c9fb8767 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:52.822479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:52.863172 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1033/2084] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:52.873087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12af58b-970a-468c-bc7c-f02c561d2576 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:52.874879 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d12af58b-970a-468c-bc7c-f02c561d2576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:52.985352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12af58b-970a-468c-bc7c-f02c561d2576 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:52.992292 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6357d5-3dc7-4340-973a-cb0589141dac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:52.993873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6357d5-3dc7-4340-973a-cb0589141dac service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:52.994175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6357d5-3dc7-4340-973a-cb0589141dac service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:52.999566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6357d5-3dc7-4340-973a-cb0589141dac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:53.044757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6357d5-3dc7-4340-973a-cb0589141dac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:53.046683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6357d5-3dc7-4340-973a-cb0589141dac service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:53.061100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1034/2085] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:53.091444 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1d2d0cd2-ffcb-4bd7-99e8-18d5c9fb8767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffba61b8eb664af5a7f968f90f9513bd', 'request_id': 'req-1d2d0cd2-ffcb-4bd7-99e8-18d5c9fb8767'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:53.097011 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6357d5-3dc7-4340-973a-cb0589141dac service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:53.098610 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4e6357d5-3dc7-4340-973a-cb0589141dac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:53.105596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1d2d0cd2-ffcb-4bd7-99e8-18d5c9fb8767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffba61b8eb664af5a7f968f90f9513bd', 'request_id': 'req-1d2d0cd2-ffcb-4bd7-99e8-18d5c9fb8767'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:53.126571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6357d5-3dc7-4340-973a-cb0589141dac service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:53.216300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1d2d0cd2-ffcb-4bd7-99e8-18d5c9fb8767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffba61b8eb664af5a7f968f90f9513bd', 'request_id': 'req-1d2d0cd2-ffcb-4bd7-99e8-18d5c9fb8767'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:53.242016 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61c917-841a-447c-8c85-1fd41b1a8024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:53.246410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1d2d0cd2-ffcb-4bd7-99e8-18d5c9fb8767 admin admin] The token cache is being invalidate because project ffba61b8eb664af5a7f968f90f9513bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:53.246772 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1d2d0cd2-ffcb-4bd7-99e8-18d5c9fb8767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:53.247389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1d2d0cd2-ffcb-4bd7-99e8-18d5c9fb8767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:53.253554 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1052/2086] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:52 2022] DELETE /identity/v3/projects/ffba61b8eb664af5a7f968f90f9513bd => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:53.254617 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61c917-841a-447c-8c85-1fd41b1a8024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:53.256342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61c917-841a-447c-8c85-1fd41b1a8024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:53.256798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61c917-841a-447c-8c85-1fd41b1a8024 admin admin] PATH_INFO: `/v3/projects/b8ef3f1e3f384c4cb3c4c267a5365c06` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:53.258811 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6fee7-fc0f-494b-a60e-07e702011dc0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:53.259108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6fee7-fc0f-494b-a60e-07e702011dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:53.259397 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6fee7-fc0f-494b-a60e-07e702011dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:53.270471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61c917-841a-447c-8c85-1fd41b1a8024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8ef3f1e3f384c4cb3c4c267a5365c06)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:53.272447 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61c917-841a-447c-8c85-1fd41b1a8024 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:53.316636 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-67f6fee7-fc0f-494b-a60e-07e702011dc0 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:53.317918 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-67f6fee7-fc0f-494b-a60e-07e702011dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:53.361200 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1053/2087] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:53.362748 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1db1d8-bd7b-4358-bf69-acbedc31f8f3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:53.365403 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1db1d8-bd7b-4358-bf69-acbedc31f8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:53.365935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1db1d8-bd7b-4358-bf69-acbedc31f8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:53.400531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ba1db1d8-bd7b-4358-bf69-acbedc31f8f3 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:53.402393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ba1db1d8-bd7b-4358-bf69-acbedc31f8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:53.434523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1054/2088] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:53.435880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8791bd6-88e7-4eeb-b09b-141a3d3346d9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:53.436278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8791bd6-88e7-4eeb-b09b-141a3d3346d9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:53.436567 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8791bd6-88e7-4eeb-b09b-141a3d3346d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:53.466761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c8791bd6-88e7-4eeb-b09b-141a3d3346d9 None None] MFA Rules not processed for user `8b983c0a2a124ac3aa3403e0777612f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:53.468152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c8791bd6-88e7-4eeb-b09b-141a3d3346d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:53.520823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cc61c917-841a-447c-8c85-1fd41b1a8024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8ef3f1e3f384c4cb3c4c267a5365c06', 'request_id': 'req-cc61c917-841a-447c-8c85-1fd41b1a8024'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:53.527127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cc61c917-841a-447c-8c85-1fd41b1a8024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8ef3f1e3f384c4cb3c4c267a5365c06', 'request_id': 'req-cc61c917-841a-447c-8c85-1fd41b1a8024'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:53.535122 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1055/2089] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:53.535472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cc61c917-841a-447c-8c85-1fd41b1a8024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8ef3f1e3f384c4cb3c4c267a5365c06', 'request_id': 'req-cc61c917-841a-447c-8c85-1fd41b1a8024'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:53.536071 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8791bd6-88e7-4eeb-b09b-141a3d3346d9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:53.555189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8791bd6-88e7-4eeb-b09b-141a3d3346d9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:53.626743 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ee57c-b32e-44e0-a11f-12026163ca24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:53.627364 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ee57c-b32e-44e0-a11f-12026163ca24 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:53.627364 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ee57c-b32e-44e0-a11f-12026163ca24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:53.627643 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ee57c-b32e-44e0-a11f-12026163ca24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:53.628206 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ee57c-b32e-44e0-a11f-12026163ca24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:53.628526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ee57c-b32e-44e0-a11f-12026163ca24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:53.630592 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ee57c-b32e-44e0-a11f-12026163ca24 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:53.645092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1056/2090] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:42:53.646074 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ee57c-b32e-44e0-a11f-12026163ca24 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:53.647721 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-577ee57c-b32e-44e0-a11f-12026163ca24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:53.670199 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ee57c-b32e-44e0-a11f-12026163ca24 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:53.677017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c854b6f-b8af-4415-a57d-8d5d460606ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:53.677772 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c854b6f-b8af-4415-a57d-8d5d460606ae admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:53.678066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c854b6f-b8af-4415-a57d-8d5d460606ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:53.678351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c854b6f-b8af-4415-a57d-8d5d460606ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:53.679112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c854b6f-b8af-4415-a57d-8d5d460606ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:53.680999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c854b6f-b8af-4415-a57d-8d5d460606ae admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:53.706134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cc61c917-841a-447c-8c85-1fd41b1a8024 admin admin] The token cache is being invalidate because project b8ef3f1e3f384c4cb3c4c267a5365c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:53.706577 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cc61c917-841a-447c-8c85-1fd41b1a8024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:53.707599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cc61c917-841a-447c-8c85-1fd41b1a8024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:53.713091 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1035/2091] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:53 2022] DELETE /identity/v3/projects/b8ef3f1e3f384c4cb3c4c267a5365c06 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:53.718085 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61c917-841a-447c-8c85-1fd41b1a8024 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:53.722472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cc61c917-841a-447c-8c85-1fd41b1a8024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:53.739904 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61c917-841a-447c-8c85-1fd41b1a8024 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:53.745979 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ec546-a5cd-4edb-b892-1b7dcca16e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:53.746991 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ec546-a5cd-4edb-b892-1b7dcca16e8e admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:53.747537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ec546-a5cd-4edb-b892-1b7dcca16e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:53.747693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ec546-a5cd-4edb-b892-1b7dcca16e8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:53.748812 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ec546-a5cd-4edb-b892-1b7dcca16e8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:53.753730 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ec546-a5cd-4edb-b892-1b7dcca16e8e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:53.771062 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1057/2092] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:53 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:53.772108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c854b6f-b8af-4415-a57d-8d5d460606ae admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:53.777714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4c854b6f-b8af-4415-a57d-8d5d460606ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:53.845215 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c854b6f-b8af-4415-a57d-8d5d460606ae admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:53.851057 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97080e15-b868-446f-a154-f0398c146c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:53.852002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97080e15-b868-446f-a154-f0398c146c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:53.853453 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97080e15-b868-446f-a154-f0398c146c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:53.853982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97080e15-b868-446f-a154-f0398c146c3b admin admin] PATH_INFO: `/v3/users/93a5a8ffb89245629058eac80dd9aaf9` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:53.856214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:53.857354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97080e15-b868-446f-a154-f0398c146c3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=93a5a8ffb89245629058eac80dd9aaf9)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:53.859647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97080e15-b868-446f-a154-f0398c146c3b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:53.868315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1036/2093] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:53 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:53.869671 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ec546-a5cd-4edb-b892-1b7dcca16e8e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:53.871643 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0f7ec546-a5cd-4edb-b892-1b7dcca16e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:53.944039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ec546-a5cd-4edb-b892-1b7dcca16e8e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:53.948728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a606f2-6c3b-44a4-b08e-7ccfae1db32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:53.949434 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a606f2-6c3b-44a4-b08e-7ccfae1db32f admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:53.949751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a606f2-6c3b-44a4-b08e-7ccfae1db32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:53.950061 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a606f2-6c3b-44a4-b08e-7ccfae1db32f admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:53.950798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a606f2-6c3b-44a4-b08e-7ccfae1db32f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:53.952734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a606f2-6c3b-44a4-b08e-7ccfae1db32f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:54.056690 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1037/2094] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:53 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:54.057729 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a606f2-6c3b-44a4-b08e-7ccfae1db32f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:54.059485 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c2a606f2-6c3b-44a4-b08e-7ccfae1db32f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:54.073306 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a606f2-6c3b-44a4-b08e-7ccfae1db32f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:54.078212 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881d2ff-b812-49f2-80e8-6b54464a73fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:54.079957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881d2ff-b812-49f2-80e8-6b54464a73fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:54.080146 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881d2ff-b812-49f2-80e8-6b54464a73fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:54.080448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881d2ff-b812-49f2-80e8-6b54464a73fd admin admin] PATH_INFO: `/v3/projects/b9cf3418aae24d11b805c3871ac18dee` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:54.081681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881d2ff-b812-49f2-80e8-6b54464a73fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9cf3418aae24d11b805c3871ac18dee)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:54.083623 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881d2ff-b812-49f2-80e8-6b54464a73fd admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:54.522390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-97080e15-b868-446f-a154-f0398c146c3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93a5a8ffb89245629058eac80dd9aaf9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:54.522681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7881d2ff-b812-49f2-80e8-6b54464a73fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9cf3418aae24d11b805c3871ac18dee', 'request_id': 'req-7881d2ff-b812-49f2-80e8-6b54464a73fd'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:54.529590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7881d2ff-b812-49f2-80e8-6b54464a73fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9cf3418aae24d11b805c3871ac18dee', 'request_id': 'req-7881d2ff-b812-49f2-80e8-6b54464a73fd'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:54.542896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-97080e15-b868-446f-a154-f0398c146c3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93a5a8ffb89245629058eac80dd9aaf9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:54.549627 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7881d2ff-b812-49f2-80e8-6b54464a73fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9cf3418aae24d11b805c3871ac18dee', 'request_id': 'req-7881d2ff-b812-49f2-80e8-6b54464a73fd'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:54.659217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-97080e15-b868-446f-a154-f0398c146c3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93a5a8ffb89245629058eac80dd9aaf9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:54.659810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:54.661232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-97080e15-b868-446f-a154-f0398c146c3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93a5a8ffb89245629058eac80dd9aaf9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:54.670813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-97080e15-b868-446f-a154-f0398c146c3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93a5a8ffb89245629058eac80dd9aaf9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:54.679356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1058/2095] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:53 2022] DELETE /identity/v3/users/93a5a8ffb89245629058eac80dd9aaf9 => generated 0 bytes in 908 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:54.680601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97080e15-b868-446f-a154-f0398c146c3b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:54.682216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-97080e15-b868-446f-a154-f0398c146c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:54.792124 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7881d2ff-b812-49f2-80e8-6b54464a73fd admin admin] The token cache is being invalidate because project b9cf3418aae24d11b805c3871ac18dee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:54.792585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7881d2ff-b812-49f2-80e8-6b54464a73fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:54.793248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-7881d2ff-b812-49f2-80e8-6b54464a73fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:54.795214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1038/2096] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:54 2022] DELETE /identity/v3/projects/b9cf3418aae24d11b805c3871ac18dee => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:54.795978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881d2ff-b812-49f2-80e8-6b54464a73fd admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:54.798086 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7881d2ff-b812-49f2-80e8-6b54464a73fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:54.803747 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97080e15-b868-446f-a154-f0398c146c3b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:54.808592 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a06a4-2ec1-42ba-84f6-793116db32d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:54.809440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a06a4-2ec1-42ba-84f6-793116db32d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:54.809791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a06a4-2ec1-42ba-84f6-793116db32d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:54.810138 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a06a4-2ec1-42ba-84f6-793116db32d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:54.810834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a06a4-2ec1-42ba-84f6-793116db32d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:54.813349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a06a4-2ec1-42ba-84f6-793116db32d2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:54.863642 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881d2ff-b812-49f2-80e8-6b54464a73fd admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:54.870123 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a250c-9976-4486-9cd3-4d22f8ac3085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:54.870509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a250c-9976-4486-9cd3-4d22f8ac3085 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:54.870785 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a250c-9976-4486-9cd3-4d22f8ac3085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:54.871731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a250c-9976-4486-9cd3-4d22f8ac3085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:54.873144 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a250c-9976-4486-9cd3-4d22f8ac3085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:54.874805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a250c-9976-4486-9cd3-4d22f8ac3085 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:54.887516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1039/2097] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:54.888915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebae2a-31db-40a3-bd1c-446987b97a60 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:54.889248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebae2a-31db-40a3-bd1c-446987b97a60 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:54.889535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebae2a-31db-40a3-bd1c-446987b97a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:54.909308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8cebae2a-31db-40a3-bd1c-446987b97a60 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:54.910354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8cebae2a-31db-40a3-bd1c-446987b97a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:54.918727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1040/2098] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:54.919775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebae2a-31db-40a3-bd1c-446987b97a60 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:54.930026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebae2a-31db-40a3-bd1c-446987b97a60 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:54.934971 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d96d20-b9f1-4b74-90fa-ffd1f4918643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:54.935846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d96d20-b9f1-4b74-90fa-ffd1f4918643 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:54.936137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d96d20-b9f1-4b74-90fa-ffd1f4918643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:54.936423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d96d20-b9f1-4b74-90fa-ffd1f4918643 admin admin] PATH_INFO: `/v3/projects/8e7b1014ab1b4cb58f1c556dd952271d/users/16a831b346e94d79925d017711a07417/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:54.951639 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d96d20-b9f1-4b74-90fa-ffd1f4918643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e7b1014ab1b4cb58f1c556dd952271d, user_id=16a831b346e94d79925d017711a07417, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:54.955010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d96d20-b9f1-4b74-90fa-ffd1f4918643 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:54.974422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1059/2099] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:54 2022] POST /identity/v3/users => generated 288 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:54.975868 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f5c05-bac1-450d-9819-16c62cb7488e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:54.976164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f5c05-bac1-450d-9819-16c62cb7488e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:54.977646 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f5c05-bac1-450d-9819-16c62cb7488e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:54.995608 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-cf5f5c05-bac1-450d-9819-16c62cb7488e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:54.996827 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cf5f5c05-bac1-450d-9819-16c62cb7488e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:55.006176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1060/2100] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:55.010492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f5c05-bac1-450d-9819-16c62cb7488e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:55.012063 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cf5f5c05-bac1-450d-9819-16c62cb7488e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:55.030536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f5c05-bac1-450d-9819-16c62cb7488e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:55.036396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0f318-c733-4719-a55d-aa2975c1e393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:55.037969 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0f318-c733-4719-a55d-aa2975c1e393 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:55.037969 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0f318-c733-4719-a55d-aa2975c1e393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:55.037969 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0f318-c733-4719-a55d-aa2975c1e393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:55.039010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0f318-c733-4719-a55d-aa2975c1e393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:55.041234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0f318-c733-4719-a55d-aa2975c1e393 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:55.053584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1061/2101] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:55.054617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0f318-c733-4719-a55d-aa2975c1e393 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:55.079911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0f318-c733-4719-a55d-aa2975c1e393 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:55.088630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf0c9c-0e2a-4b54-8e64-a6fb4b02a21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:55.090561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf0c9c-0e2a-4b54-8e64-a6fb4b02a21f admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:55.091298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf0c9c-0e2a-4b54-8e64-a6fb4b02a21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:55.091835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf0c9c-0e2a-4b54-8e64-a6fb4b02a21f admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:55.093679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf0c9c-0e2a-4b54-8e64-a6fb4b02a21f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:55.093679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf0c9c-0e2a-4b54-8e64-a6fb4b02a21f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:55.098590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf0c9c-0e2a-4b54-8e64-a6fb4b02a21f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:55.103173 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1041/2102] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:54 2022] PUT /identity/v3/projects/8e7b1014ab1b4cb58f1c556dd952271d/users/16a831b346e94d79925d017711a07417/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:55.104851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d96d20-b9f1-4b74-90fa-ffd1f4918643 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:55.121641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1062/2103] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:42:55.121641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf0c9c-0e2a-4b54-8e64-a6fb4b02a21f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:55.148049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf0c9c-0e2a-4b54-8e64-a6fb4b02a21f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:55.270140 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9e7f6-96ac-482a-b577-bd319c02d672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:55.270903 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9e7f6-96ac-482a-b577-bd319c02d672 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:55.271236 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9e7f6-96ac-482a-b577-bd319c02d672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:55.271570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9e7f6-96ac-482a-b577-bd319c02d672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:55.272369 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9e7f6-96ac-482a-b577-bd319c02d672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:55.274374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9e7f6-96ac-482a-b577-bd319c02d672 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:55.274784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d96d20-b9f1-4b74-90fa-ffd1f4918643 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:55.289235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb834c-6946-4a74-bb71-f439fd8ad890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:55.290367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb834c-6946-4a74-bb71-f439fd8ad890 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:55.290977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb834c-6946-4a74-bb71-f439fd8ad890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:55.291441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb834c-6946-4a74-bb71-f439fd8ad890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:55.292431 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb834c-6946-4a74-bb71-f439fd8ad890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:55.292557 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1063/2104] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:55.293394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9e7f6-96ac-482a-b577-bd319c02d672 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:55.295202 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb834c-6946-4a74-bb71-f439fd8ad890 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:55.306642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9e7f6-96ac-482a-b577-bd319c02d672 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:55.309309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1042/2105] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:55.313228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311664a4-031c-43c9-8f8b-2dd4f2e6128e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:55.313673 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311664a4-031c-43c9-8f8b-2dd4f2e6128e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:55.313966 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311664a4-031c-43c9-8f8b-2dd4f2e6128e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:55.314252 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311664a4-031c-43c9-8f8b-2dd4f2e6128e admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:55.315069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311664a4-031c-43c9-8f8b-2dd4f2e6128e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:55.315622 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb834c-6946-4a74-bb71-f439fd8ad890 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:55.317159 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311664a4-031c-43c9-8f8b-2dd4f2e6128e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:55.332039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb834c-6946-4a74-bb71-f439fd8ad890 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:55.339983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09ec52-c644-4e61-b727-870e1b4c387c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:55.341404 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09ec52-c644-4e61-b727-870e1b4c387c admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:55.341839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09ec52-c644-4e61-b727-870e1b4c387c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:55.342536 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09ec52-c644-4e61-b727-870e1b4c387c admin admin] PATH_INFO: `/v3/projects/8e7b1014ab1b4cb58f1c556dd952271d/users/16a831b346e94d79925d017711a07417/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:55.345708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09ec52-c644-4e61-b727-870e1b4c387c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e7b1014ab1b4cb58f1c556dd952271d, user_id=16a831b346e94d79925d017711a07417, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:55.349203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09ec52-c644-4e61-b727-870e1b4c387c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:55.360762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1043/2106] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:55 2022] PUT /identity/v3/projects/8e7b1014ab1b4cb58f1c556dd952271d/users/16a831b346e94d79925d017711a07417/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:55.361947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09ec52-c644-4e61-b727-870e1b4c387c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:55.431665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09ec52-c644-4e61-b727-870e1b4c387c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:55.436670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699e741-d870-4eea-a7da-124d2a311bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:55.438132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699e741-d870-4eea-a7da-124d2a311bf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:55.438602 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699e741-d870-4eea-a7da-124d2a311bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:55.438890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699e741-d870-4eea-a7da-124d2a311bf8 admin admin] PATH_INFO: `/v3/projects/4e0071e2898841c8aceb9086db240a25/users/961df89ea9fb46f397589baf93ea0335/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:55.448685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1064/2107] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:55 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:55.454446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311664a4-031c-43c9-8f8b-2dd4f2e6128e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:55.471089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699e741-d870-4eea-a7da-124d2a311bf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e0071e2898841c8aceb9086db240a25, user_id=961df89ea9fb46f397589baf93ea0335, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:55.474319 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699e741-d870-4eea-a7da-124d2a311bf8 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:55.580426 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311664a4-031c-43c9-8f8b-2dd4f2e6128e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:55.590672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1caa6ea-b588-45ad-a2db-234c9dccf9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:55.591708 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1caa6ea-b588-45ad-a2db-234c9dccf9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:55.592202 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1caa6ea-b588-45ad-a2db-234c9dccf9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:55.592792 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1caa6ea-b588-45ad-a2db-234c9dccf9b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:55.593724 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1caa6ea-b588-45ad-a2db-234c9dccf9b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:55.596111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1caa6ea-b588-45ad-a2db-234c9dccf9b8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:55.624752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1044/2108] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:55 2022] PUT /identity/v3/projects/4e0071e2898841c8aceb9086db240a25/users/961df89ea9fb46f397589baf93ea0335/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:55.624752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699e741-d870-4eea-a7da-124d2a311bf8 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:55.631190 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a699e741-d870-4eea-a7da-124d2a311bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:55.684872 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1065/2109] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:55 2022] POST /identity/v3/users => generated 287 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:55.687190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984f921-7630-4dbd-bb2e-3cd09042bb77 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:55.688380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984f921-7630-4dbd-bb2e-3cd09042bb77 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:55.688941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984f921-7630-4dbd-bb2e-3cd09042bb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:55.742209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d984f921-7630-4dbd-bb2e-3cd09042bb77 None None] MFA Rules not processed for user `961df89ea9fb46f397589baf93ea0335`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:55.743820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d984f921-7630-4dbd-bb2e-3cd09042bb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:55.765124 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699e741-d870-4eea-a7da-124d2a311bf8 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:55.774780 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1d070-a079-4730-88a2-c1f8588facd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:55.775729 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1d070-a079-4730-88a2-c1f8588facd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:55.776021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1d070-a079-4730-88a2-c1f8588facd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:55.776301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1d070-a079-4730-88a2-c1f8588facd4 admin admin] PATH_INFO: `/v3/projects/38fd6f23aff249658b7b883c9bea9984` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:55.779764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1d070-a079-4730-88a2-c1f8588facd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38fd6f23aff249658b7b883c9bea9984)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:55.782902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1d070-a079-4730-88a2-c1f8588facd4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:55.852205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1066/2110] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:55.853246 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984f921-7630-4dbd-bb2e-3cd09042bb77 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:55.867893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984f921-7630-4dbd-bb2e-3cd09042bb77 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:55.885729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d115a4-7166-4205-93c0-8ce59cdf53b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:55.885729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d115a4-7166-4205-93c0-8ce59cdf53b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:55.885729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d115a4-7166-4205-93c0-8ce59cdf53b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:55.885729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d115a4-7166-4205-93c0-8ce59cdf53b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:55.893807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d115a4-7166-4205-93c0-8ce59cdf53b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:55.895650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d115a4-7166-4205-93c0-8ce59cdf53b7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:55.930274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1067/2111] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:55.931169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d115a4-7166-4205-93c0-8ce59cdf53b7 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:55.950614 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d115a4-7166-4205-93c0-8ce59cdf53b7 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:55.951876 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-58a1d070-a079-4730-88a2-c1f8588facd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38fd6f23aff249658b7b883c9bea9984', 'request_id': 'req-58a1d070-a079-4730-88a2-c1f8588facd4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:55.977323 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-58a1d070-a079-4730-88a2-c1f8588facd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38fd6f23aff249658b7b883c9bea9984', 'request_id': 'req-58a1d070-a079-4730-88a2-c1f8588facd4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:55.990661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-58a1d070-a079-4730-88a2-c1f8588facd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38fd6f23aff249658b7b883c9bea9984', 'request_id': 'req-58a1d070-a079-4730-88a2-c1f8588facd4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:56.090393 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-58a1d070-a079-4730-88a2-c1f8588facd4 admin admin] The token cache is being invalidate because project 38fd6f23aff249658b7b883c9bea9984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:56.091423 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-58a1d070-a079-4730-88a2-c1f8588facd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:56.093117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-58a1d070-a079-4730-88a2-c1f8588facd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:56.099164 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1045/2112] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:55 2022] DELETE /identity/v3/projects/38fd6f23aff249658b7b883c9bea9984 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:56.099730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f959a-1f6b-46bc-8db2-bb24fb227eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:56.100858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1d070-a079-4730-88a2-c1f8588facd4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:56.101427 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f959a-1f6b-46bc-8db2-bb24fb227eaf admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:56.101714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f959a-1f6b-46bc-8db2-bb24fb227eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:56.102031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f959a-1f6b-46bc-8db2-bb24fb227eaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:56.103988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-58a1d070-a079-4730-88a2-c1f8588facd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:56.105818 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f959a-1f6b-46bc-8db2-bb24fb227eaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:56.112442 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f959a-1f6b-46bc-8db2-bb24fb227eaf admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:56.143577 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1d070-a079-4730-88a2-c1f8588facd4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:56.160351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ceabf5-1f5b-41c2-99cb-9aaa98c69135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:56.161397 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ceabf5-1f5b-41c2-99cb-9aaa98c69135 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:56.162005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ceabf5-1f5b-41c2-99cb-9aaa98c69135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:56.162661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ceabf5-1f5b-41c2-99cb-9aaa98c69135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:56.163894 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ceabf5-1f5b-41c2-99cb-9aaa98c69135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:56.167150 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ceabf5-1f5b-41c2-99cb-9aaa98c69135 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:56.186487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1046/2113] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:56.190670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358791b-07c1-46d2-8972-7101a3c55465 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:56.191176 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358791b-07c1-46d2-8972-7101a3c55465 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:56.191709 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358791b-07c1-46d2-8972-7101a3c55465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:56.228603 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1068/2114] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:55 2022] POST /identity/v3/projects => generated 375 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:56.232739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f959a-1f6b-46bc-8db2-bb24fb227eaf admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:56.233591 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2358791b-07c1-46d2-8972-7101a3c55465 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:56.238394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2358791b-07c1-46d2-8972-7101a3c55465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:56.346572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1047/2115] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:56.347761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358791b-07c1-46d2-8972-7101a3c55465 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:56.349288 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2358791b-07c1-46d2-8972-7101a3c55465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:56.350790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f959a-1f6b-46bc-8db2-bb24fb227eaf admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:56.356203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefeaf3-bbe6-423a-81fd-94c43d2d41c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:56.360649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefeaf3-bbe6-423a-81fd-94c43d2d41c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:56.360649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefeaf3-bbe6-423a-81fd-94c43d2d41c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:56.360649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefeaf3-bbe6-423a-81fd-94c43d2d41c4 admin admin] PATH_INFO: `/v3/projects/c2ef57ea448a49d3af268a02f6f08257/users/6badcf8b92dc43a3bc2342e303dec1da/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:56.367597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358791b-07c1-46d2-8972-7101a3c55465 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:56.374399 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77dcc6-2f5d-4558-9fa9-f5528fd3b26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:56.375243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77dcc6-2f5d-4558-9fa9-f5528fd3b26e admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:56.375681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77dcc6-2f5d-4558-9fa9-f5528fd3b26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:56.376098 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77dcc6-2f5d-4558-9fa9-f5528fd3b26e admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:56.377294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77dcc6-2f5d-4558-9fa9-f5528fd3b26e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:56.382432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77dcc6-2f5d-4558-9fa9-f5528fd3b26e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:56.399093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefeaf3-bbe6-423a-81fd-94c43d2d41c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ef57ea448a49d3af268a02f6f08257, user_id=6badcf8b92dc43a3bc2342e303dec1da, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:56.402803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefeaf3-bbe6-423a-81fd-94c43d2d41c4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:56.479545 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:56.487753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1048/2116] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:56 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:56.490897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a0a0f-21ce-4a2b-9829-96fd0389d44f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:56.491497 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a0a0f-21ce-4a2b-9829-96fd0389d44f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:56.492348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a0a0f-21ce-4a2b-9829-96fd0389d44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:56.517551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e48a0a0f-21ce-4a2b-9829-96fd0389d44f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:56.528069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e48a0a0f-21ce-4a2b-9829-96fd0389d44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:56.542501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1049/2117] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:56.542844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a0a0f-21ce-4a2b-9829-96fd0389d44f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:56.551862 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e48a0a0f-21ce-4a2b-9829-96fd0389d44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:56.564493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1069/2118] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:56 2022] PUT /identity/v3/projects/c2ef57ea448a49d3af268a02f6f08257/users/6badcf8b92dc43a3bc2342e303dec1da/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:56.565816 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefeaf3-bbe6-423a-81fd-94c43d2d41c4 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:56.581720 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a0a0f-21ce-4a2b-9829-96fd0389d44f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:56.655046 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefeaf3-bbe6-423a-81fd-94c43d2d41c4 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:56.665194 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ededc-9d3c-458a-bd63-431f979a4372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:56.667905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ededc-9d3c-458a-bd63-431f979a4372 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:56.668516 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ededc-9d3c-458a-bd63-431f979a4372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:56.669002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ededc-9d3c-458a-bd63-431f979a4372 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:56.670342 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ededc-9d3c-458a-bd63-431f979a4372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:56.673919 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ededc-9d3c-458a-bd63-431f979a4372 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:56.679970 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070990a8-bc0c-48f9-98dd-f2477ca6c783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:56.681330 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070990a8-bc0c-48f9-98dd-f2477ca6c783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:56.681616 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070990a8-bc0c-48f9-98dd-f2477ca6c783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:56.681895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070990a8-bc0c-48f9-98dd-f2477ca6c783 admin admin] PATH_INFO: `/v3/users/a60bfca1aff144f385fdd23a44f2fcc8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:56.683328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070990a8-bc0c-48f9-98dd-f2477ca6c783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a60bfca1aff144f385fdd23a44f2fcc8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:56.685349 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070990a8-bc0c-48f9-98dd-f2477ca6c783 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:56.695335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1070/2119] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:56.695335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ededc-9d3c-458a-bd63-431f979a4372 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:56.722289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ededc-9d3c-458a-bd63-431f979a4372 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:56.728690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbd6f7-c8b4-4608-abe5-65843447f42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:56.729462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbd6f7-c8b4-4608-abe5-65843447f42c admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:56.729752 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbd6f7-c8b4-4608-abe5-65843447f42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:56.730032 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbd6f7-c8b4-4608-abe5-65843447f42c admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:56.730939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbd6f7-c8b4-4608-abe5-65843447f42c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:42:56.731413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbd6f7-c8b4-4608-abe5-65843447f42c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:56.733415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbd6f7-c8b4-4608-abe5-65843447f42c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:56.748713 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1071/2120] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:42:56.750196 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ed04c-823c-4817-bd83-9bb2d72e14e5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:56.750594 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ed04c-823c-4817-bd83-9bb2d72e14e5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:56.751181 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ed04c-823c-4817-bd83-9bb2d72e14e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:56.771079 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ab0ed04c-823c-4817-bd83-9bb2d72e14e5 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:56.772393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ab0ed04c-823c-4817-bd83-9bb2d72e14e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:56.780611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1072/2121] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:56.785797 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ed04c-823c-4817-bd83-9bb2d72e14e5 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:56.796943 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ed04c-823c-4817-bd83-9bb2d72e14e5 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:56.804214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bb146-0398-44de-b6b6-33d491630a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:56.804214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bb146-0398-44de-b6b6-33d491630a43 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:56.804214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bb146-0398-44de-b6b6-33d491630a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:56.804552 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bb146-0398-44de-b6b6-33d491630a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:56.805250 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9bb146-0398-44de-b6b6-33d491630a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:56.806879 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9bb146-0398-44de-b6b6-33d491630a43 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:56.822386 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1073/2122] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:56.822386 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bb146-0398-44de-b6b6-33d491630a43 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:56.834445 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bb146-0398-44de-b6b6-33d491630a43 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:56.846192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345ba52-f4b7-401b-81fa-49da37408017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:56.846192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345ba52-f4b7-401b-81fa-49da37408017 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:56.846192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345ba52-f4b7-401b-81fa-49da37408017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:56.846192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345ba52-f4b7-401b-81fa-49da37408017 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:56.846658 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345ba52-f4b7-401b-81fa-49da37408017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:56.846658 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345ba52-f4b7-401b-81fa-49da37408017 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:56.911907 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-070990a8-bc0c-48f9-98dd-f2477ca6c783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a60bfca1aff144f385fdd23a44f2fcc8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:56.912855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-070990a8-bc0c-48f9-98dd-f2477ca6c783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a60bfca1aff144f385fdd23a44f2fcc8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:56.916517 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-070990a8-bc0c-48f9-98dd-f2477ca6c783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a60bfca1aff144f385fdd23a44f2fcc8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:56.927356 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-070990a8-bc0c-48f9-98dd-f2477ca6c783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a60bfca1aff144f385fdd23a44f2fcc8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:56.942834 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-070990a8-bc0c-48f9-98dd-f2477ca6c783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a60bfca1aff144f385fdd23a44f2fcc8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:56.994660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1074/2123] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:56 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:56.995624 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345ba52-f4b7-401b-81fa-49da37408017 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:56.997587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4345ba52-f4b7-401b-81fa-49da37408017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:57.021662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1050/2124] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:56 2022] DELETE /identity/v3/users/a60bfca1aff144f385fdd23a44f2fcc8 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:57.021662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070990a8-bc0c-48f9-98dd-f2477ca6c783 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:57.029785 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-070990a8-bc0c-48f9-98dd-f2477ca6c783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:57.073880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345ba52-f4b7-401b-81fa-49da37408017 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:57.106513 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070990a8-bc0c-48f9-98dd-f2477ca6c783 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:57.111896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:57.113468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d39ad5-2e5c-4173-b8f2-9eaa376acc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:57.114384 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d39ad5-2e5c-4173-b8f2-9eaa376acc99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:57.114672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d39ad5-2e5c-4173-b8f2-9eaa376acc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:57.114956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d39ad5-2e5c-4173-b8f2-9eaa376acc99 admin admin] PATH_INFO: `/v3/projects/b42e249de8fb4db3be4f7777d9dbb00c/users/28553b6adc6c42c29a1bec131a44968b/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:57.149533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4ceae-2372-4e63-9f12-31888f1ef677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:57.150568 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4ceae-2372-4e63-9f12-31888f1ef677 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:57.151069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4ceae-2372-4e63-9f12-31888f1ef677 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:57.151821 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4ceae-2372-4e63-9f12-31888f1ef677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:57.158534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8fa4ceae-2372-4e63-9f12-31888f1ef677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:57.166752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d39ad5-2e5c-4173-b8f2-9eaa376acc99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42e249de8fb4db3be4f7777d9dbb00c, user_id=28553b6adc6c42c29a1bec131a44968b, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:57.173515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d39ad5-2e5c-4173-b8f2-9eaa376acc99 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:57.178564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4ceae-2372-4e63-9f12-31888f1ef677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:57.180285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4ceae-2372-4e63-9f12-31888f1ef677 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:57.191685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1075/2125] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:57.192636 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4ceae-2372-4e63-9f12-31888f1ef677 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:57.194492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8fa4ceae-2372-4e63-9f12-31888f1ef677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:57.211354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4ceae-2372-4e63-9f12-31888f1ef677 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:57.216610 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347eeb5e-5cfc-4a5d-a121-11b5e176d931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:57.217424 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347eeb5e-5cfc-4a5d-a121-11b5e176d931 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:57.217804 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347eeb5e-5cfc-4a5d-a121-11b5e176d931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:57.218158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347eeb5e-5cfc-4a5d-a121-11b5e176d931 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:57.219533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347eeb5e-5cfc-4a5d-a121-11b5e176d931 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:57.222205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347eeb5e-5cfc-4a5d-a121-11b5e176d931 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:57.259472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1051/2126] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:57 2022] PUT /identity/v3/projects/b42e249de8fb4db3be4f7777d9dbb00c/users/28553b6adc6c42c29a1bec131a44968b/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:57.260962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3483425-ac3a-4151-85a9-b365ab0ea7b2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:57.261292 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3483425-ac3a-4151-85a9-b365ab0ea7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:57.263619 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3483425-ac3a-4151-85a9-b365ab0ea7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:57.284886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a3483425-ac3a-4151-85a9-b365ab0ea7b2 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:57.286274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a3483425-ac3a-4151-85a9-b365ab0ea7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:57.363961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1076/2127] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:57 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:57.365046 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347eeb5e-5cfc-4a5d-a121-11b5e176d931 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:57.368947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-347eeb5e-5cfc-4a5d-a121-11b5e176d931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:57.386287 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1052/2128] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:57.388004 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89034a53-872f-4098-9fbc-70504e9dabac None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:57.388307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89034a53-872f-4098-9fbc-70504e9dabac None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:57.388592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89034a53-872f-4098-9fbc-70504e9dabac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:57.408693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347eeb5e-5cfc-4a5d-a121-11b5e176d931 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:57.421305 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c15155-0d1b-4498-9e2d-e489fb339a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:57.422332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c15155-0d1b-4498-9e2d-e489fb339a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:57.422691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c15155-0d1b-4498-9e2d-e489fb339a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:57.423307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c15155-0d1b-4498-9e2d-e489fb339a69 admin admin] PATH_INFO: `/v3/users/916c7e4c45a94167ba39db8c7fec0d19` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:57.425890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c15155-0d1b-4498-9e2d-e489fb339a69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=916c7e4c45a94167ba39db8c7fec0d19)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:57.427934 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c15155-0d1b-4498-9e2d-e489fb339a69 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:57.440444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-89034a53-872f-4098-9fbc-70504e9dabac None None] MFA Rules not processed for user `28553b6adc6c42c29a1bec131a44968b`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:57.446292 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-89034a53-872f-4098-9fbc-70504e9dabac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:57.555630 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1053/2129] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:57.559141 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89034a53-872f-4098-9fbc-70504e9dabac None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:57.577091 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89034a53-872f-4098-9fbc-70504e9dabac None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:57.577920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ea39c-1c4e-4b71-a130-ce36d439e039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:57.579544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ea39c-1c4e-4b71-a130-ce36d439e039 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:57.579848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ea39c-1c4e-4b71-a130-ce36d439e039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:57.580135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ea39c-1c4e-4b71-a130-ce36d439e039 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:57.580886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ea39c-1c4e-4b71-a130-ce36d439e039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:57.582594 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ea39c-1c4e-4b71-a130-ce36d439e039 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:57.602954 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1054/2130] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:57.607156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ea39c-1c4e-4b71-a130-ce36d439e039 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:57.673906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ea39c-1c4e-4b71-a130-ce36d439e039 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:57.677997 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6b96f-d88d-4f20-af38-b9a2c4cc624c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:57.678905 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6b96f-d88d-4f20-af38-b9a2c4cc624c service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:57.679266 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6b96f-d88d-4f20-af38-b9a2c4cc624c service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:57.679649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6b96f-d88d-4f20-af38-b9a2c4cc624c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:57.702258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6b96f-d88d-4f20-af38-b9a2c4cc624c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:57.706488 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6b96f-d88d-4f20-af38-b9a2c4cc624c service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:57.726134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1055/2131] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:57.727539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6b96f-d88d-4f20-af38-b9a2c4cc624c service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:57.734512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d2c15155-0d1b-4498-9e2d-e489fb339a69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '916c7e4c45a94167ba39db8c7fec0d19'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:57.744090 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6b96f-d88d-4f20-af38-b9a2c4cc624c service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:57.751523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6229c-2ae2-4f38-bb35-fb8bd0b2fa73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:57.753345 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6229c-2ae2-4f38-bb35-fb8bd0b2fa73 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:57.753750 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6229c-2ae2-4f38-bb35-fb8bd0b2fa73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:57.754107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6229c-2ae2-4f38-bb35-fb8bd0b2fa73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:57.755702 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6229c-2ae2-4f38-bb35-fb8bd0b2fa73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:57.758485 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6229c-2ae2-4f38-bb35-fb8bd0b2fa73 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:57.767119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d2c15155-0d1b-4498-9e2d-e489fb339a69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '916c7e4c45a94167ba39db8c7fec0d19'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:57.841831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d2c15155-0d1b-4498-9e2d-e489fb339a69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '916c7e4c45a94167ba39db8c7fec0d19'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:57.842467 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d2c15155-0d1b-4498-9e2d-e489fb339a69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '916c7e4c45a94167ba39db8c7fec0d19'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:57.851725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d2c15155-0d1b-4498-9e2d-e489fb339a69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '916c7e4c45a94167ba39db8c7fec0d19'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:57.870967 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1077/2132] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:57 2022] DELETE /identity/v3/users/916c7e4c45a94167ba39db8c7fec0d19 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:57.872500 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c15155-0d1b-4498-9e2d-e489fb339a69 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:57.879486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d2c15155-0d1b-4498-9e2d-e489fb339a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:57.879607 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1056/2133] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:42:57 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:42:57.880775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6229c-2ae2-4f38-bb35-fb8bd0b2fa73 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:57.882784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5cc6229c-2ae2-4f38-bb35-fb8bd0b2fa73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:58.004167 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c15155-0d1b-4498-9e2d-e489fb339a69 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:58.013347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f88b84-129f-4108-88a2-60aada878113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:58.014112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f88b84-129f-4108-88a2-60aada878113 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:58.014477 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f88b84-129f-4108-88a2-60aada878113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:58.014929 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f88b84-129f-4108-88a2-60aada878113 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:58.015922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f88b84-129f-4108-88a2-60aada878113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:58.017961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f88b84-129f-4108-88a2-60aada878113 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:58.025672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6229c-2ae2-4f38-bb35-fb8bd0b2fa73 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:58.041899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1078/2134] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:58.046105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f88b84-129f-4108-88a2-60aada878113 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:58.046267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86561e1a-d122-49a9-b4e6-69355d11f765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:58.047459 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86561e1a-d122-49a9-b4e6-69355d11f765 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:58.047761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86561e1a-d122-49a9-b4e6-69355d11f765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:58.048047 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86561e1a-d122-49a9-b4e6-69355d11f765 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:58.048759 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86561e1a-d122-49a9-b4e6-69355d11f765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:58.050972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86561e1a-d122-49a9-b4e6-69355d11f765 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:58.073115 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f88b84-129f-4108-88a2-60aada878113 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:58.090114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f21be3-0f31-42e3-bc4d-bb92d2af31c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:58.091038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f21be3-0f31-42e3-bc4d-bb92d2af31c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:58.091321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f21be3-0f31-42e3-bc4d-bb92d2af31c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:58.097399 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f21be3-0f31-42e3-bc4d-bb92d2af31c6 admin admin] PATH_INFO: `/v3/projects/bb1b995c217142cc9bd40c90e2599888/users/0f9eb51be45749ba867402297051014f/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:58.122888 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f21be3-0f31-42e3-bc4d-bb92d2af31c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb1b995c217142cc9bd40c90e2599888, user_id=0f9eb51be45749ba867402297051014f, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:58.128392 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f21be3-0f31-42e3-bc4d-bb92d2af31c6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:58.157438 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1057/2135] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:42:57 2022] POST /identity/v3/users => generated 295 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:42:58.162875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86561e1a-d122-49a9-b4e6-69355d11f765 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:58.179378 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86561e1a-d122-49a9-b4e6-69355d11f765 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:58.185873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fed7a-fed9-47eb-87b3-1b00f82b48c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:58.187222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fed7a-fed9-47eb-87b3-1b00f82b48c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:58.189285 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fed7a-fed9-47eb-87b3-1b00f82b48c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:58.189727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fed7a-fed9-47eb-87b3-1b00f82b48c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:58.190647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fed7a-fed9-47eb-87b3-1b00f82b48c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:58.194197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fed7a-fed9-47eb-87b3-1b00f82b48c2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:58.208866 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1058/2136] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:42:58.212926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fed7a-fed9-47eb-87b3-1b00f82b48c2 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:58.225630 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fed7a-fed9-47eb-87b3-1b00f82b48c2 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:58.228404 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1079/2137] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:58 2022] PUT /identity/v3/projects/bb1b995c217142cc9bd40c90e2599888/users/0f9eb51be45749ba867402297051014f/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:58.242036 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fbe98-fd1b-4149-869a-e7d565e8b3f9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:58.242476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fbe98-fd1b-4149-869a-e7d565e8b3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:58.242862 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fbe98-fd1b-4149-869a-e7d565e8b3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:58.304001 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54954ed2-27a7-47f5-925a-ffcae059fe9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:58.305107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54954ed2-27a7-47f5-925a-ffcae059fe9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:58.305221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54954ed2-27a7-47f5-925a-ffcae059fe9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:58.305545 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54954ed2-27a7-47f5-925a-ffcae059fe9d admin admin] PATH_INFO: `/v3/projects/0edf674f1f16448b895cd354efdc6406/users/e2a9a8df58bf4856a50387bf50da623d/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:58.322737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54954ed2-27a7-47f5-925a-ffcae059fe9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0edf674f1f16448b895cd354efdc6406, user_id=e2a9a8df58bf4856a50387bf50da623d, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:58.325496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54954ed2-27a7-47f5-925a-ffcae059fe9d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:58.327269 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c96fbe98-fd1b-4149-869a-e7d565e8b3f9 None None] MFA Rules not processed for user `0f9eb51be45749ba867402297051014f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:58.329685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c96fbe98-fd1b-4149-869a-e7d565e8b3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:58.446460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1059/2138] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:58 2022] PUT /identity/v3/projects/0edf674f1f16448b895cd354efdc6406/users/e2a9a8df58bf4856a50387bf50da623d/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:42:58.450390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54954ed2-27a7-47f5-925a-ffcae059fe9d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:58.467780 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1080/2139] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:58.488269 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fbe98-fd1b-4149-869a-e7d565e8b3f9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:58.491837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c96fbe98-fd1b-4149-869a-e7d565e8b3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:58.587615 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54954ed2-27a7-47f5-925a-ffcae059fe9d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:58.591651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fbe98-fd1b-4149-869a-e7d565e8b3f9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:58.593301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d06b5-ed6a-4a6d-ac0f-205ae7aefc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:58.594180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d06b5-ed6a-4a6d-ac0f-205ae7aefc70 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:58.594754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d06b5-ed6a-4a6d-ac0f-205ae7aefc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:58.594754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d06b5-ed6a-4a6d-ac0f-205ae7aefc70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:58.595696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d06b5-ed6a-4a6d-ac0f-205ae7aefc70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:58.597334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d06b5-ed6a-4a6d-ac0f-205ae7aefc70 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:58.612337 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1060/2140] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:42:58.616550 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d06b5-ed6a-4a6d-ac0f-205ae7aefc70 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:58.657919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d06b5-ed6a-4a6d-ac0f-205ae7aefc70 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:58.678443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492272b-9d77-484b-a67f-0b8eb213f16e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:58.679281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492272b-9d77-484b-a67f-0b8eb213f16e service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:58.679961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492272b-9d77-484b-a67f-0b8eb213f16e service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:58.679961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492272b-9d77-484b-a67f-0b8eb213f16e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:58.687738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4054710-0e64-4572-b34f-1b4bc13448d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:58.689567 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4054710-0e64-4572-b34f-1b4bc13448d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:58.690045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4054710-0e64-4572-b34f-1b4bc13448d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:58.690499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4054710-0e64-4572-b34f-1b4bc13448d9 admin admin] PATH_INFO: `/v3/projects/0edf674f1f16448b895cd354efdc6406/users/e2a9a8df58bf4856a50387bf50da623d/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:58.693510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4054710-0e64-4572-b34f-1b4bc13448d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0edf674f1f16448b895cd354efdc6406, user_id=e2a9a8df58bf4856a50387bf50da623d, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:58.696734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4054710-0e64-4572-b34f-1b4bc13448d9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:58.768836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0492272b-9d77-484b-a67f-0b8eb213f16e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:58.770382 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0492272b-9d77-484b-a67f-0b8eb213f16e service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:58.781182 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1061/2141] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:42:58 2022] PUT /identity/v3/projects/0edf674f1f16448b895cd354efdc6406/users/e2a9a8df58bf4856a50387bf50da623d/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:58.781182 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4054710-0e64-4572-b34f-1b4bc13448d9 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:58.785685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1081/2142] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:42:58.789437 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b4054710-0e64-4572-b34f-1b4bc13448d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:58.795412 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492272b-9d77-484b-a67f-0b8eb213f16e service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:58.799230 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0492272b-9d77-484b-a67f-0b8eb213f16e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:58.818014 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:58.846517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492272b-9d77-484b-a67f-0b8eb213f16e service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:58.925641 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4054710-0e64-4572-b34f-1b4bc13448d9 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:58.930972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a315b-6a13-44cc-b03f-70e8e3061480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:58.932444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a315b-6a13-44cc-b03f-70e8e3061480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:58.932575 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a315b-6a13-44cc-b03f-70e8e3061480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:58.932832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a315b-6a13-44cc-b03f-70e8e3061480 admin admin] PATH_INFO: `/v3/projects/0a819820263647f784079581c7527921` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:58.934107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a315b-6a13-44cc-b03f-70e8e3061480 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a819820263647f784079581c7527921)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:58.936298 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a315b-6a13-44cc-b03f-70e8e3061480 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:58.968394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd628374-42cc-4dfe-b2cf-d9732f423c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:58.972266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd628374-42cc-4dfe-b2cf-d9732f423c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:58.972801 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd628374-42cc-4dfe-b2cf-d9732f423c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:58.973448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd628374-42cc-4dfe-b2cf-d9732f423c12 admin admin] PATH_INFO: `/v3/projects/41af6594a19944c3a7bdcb88856b4a49` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:58.975018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd628374-42cc-4dfe-b2cf-d9732f423c12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41af6594a19944c3a7bdcb88856b4a49)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:58.977545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd628374-42cc-4dfe-b2cf-d9732f423c12 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:59.070542 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:59.107239 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dd628374-42cc-4dfe-b2cf-d9732f423c12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41af6594a19944c3a7bdcb88856b4a49', 'request_id': 'req-dd628374-42cc-4dfe-b2cf-d9732f423c12'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:59.110220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c03a315b-6a13-44cc-b03f-70e8e3061480 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a819820263647f784079581c7527921', 'request_id': 'req-c03a315b-6a13-44cc-b03f-70e8e3061480'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:59.120202 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dd628374-42cc-4dfe-b2cf-d9732f423c12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41af6594a19944c3a7bdcb88856b4a49', 'request_id': 'req-dd628374-42cc-4dfe-b2cf-d9732f423c12'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:59.126112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c03a315b-6a13-44cc-b03f-70e8e3061480 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a819820263647f784079581c7527921', 'request_id': 'req-c03a315b-6a13-44cc-b03f-70e8e3061480'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:59.146916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c03a315b-6a13-44cc-b03f-70e8e3061480 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a819820263647f784079581c7527921', 'request_id': 'req-c03a315b-6a13-44cc-b03f-70e8e3061480'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:59.202266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dd628374-42cc-4dfe-b2cf-d9732f423c12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41af6594a19944c3a7bdcb88856b4a49', 'request_id': 'req-dd628374-42cc-4dfe-b2cf-d9732f423c12'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:59.235086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dd628374-42cc-4dfe-b2cf-d9732f423c12 admin admin] The token cache is being invalidate because project 41af6594a19944c3a7bdcb88856b4a49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:59.235942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dd628374-42cc-4dfe-b2cf-d9732f423c12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:59.236575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dd628374-42cc-4dfe-b2cf-d9732f423c12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:59.238843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1082/2143] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:58 2022] DELETE /identity/v3/projects/41af6594a19944c3a7bdcb88856b4a49 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:59.240327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e60623-74d4-4f95-b951-0aea973a9e76 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:59.240720 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e60623-74d4-4f95-b951-0aea973a9e76 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:59.241091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e60623-74d4-4f95-b951-0aea973a9e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:59.287811 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-53e60623-74d4-4f95-b951-0aea973a9e76 None None] MFA Rules not processed for user `e2a9a8df58bf4856a50387bf50da623d`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:59.288764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-53e60623-74d4-4f95-b951-0aea973a9e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:59.295488 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c03a315b-6a13-44cc-b03f-70e8e3061480 admin admin] The token cache is being invalidate because project 0a819820263647f784079581c7527921 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:42:59.296326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c03a315b-6a13-44cc-b03f-70e8e3061480 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:59.298511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c03a315b-6a13-44cc-b03f-70e8e3061480 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:59.309674 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1062/2144] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:42:58 2022] DELETE /identity/v3/projects/0a819820263647f784079581c7527921 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:42:59.310693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a315b-6a13-44cc-b03f-70e8e3061480 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:59.330019 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c03a315b-6a13-44cc-b03f-70e8e3061480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:59.441232 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1083/2145] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:59.444173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff2058-f7b3-400d-88b1-13396d813926 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:59.444724 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff2058-f7b3-400d-88b1-13396d813926 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:59.445581 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff2058-f7b3-400d-88b1-13396d813926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:59.528668 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f2ff2058-f7b3-400d-88b1-13396d813926 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:59.532240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f2ff2058-f7b3-400d-88b1-13396d813926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:59.562667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1084/2146] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:59.564556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15319baa-1a53-419b-9e8d-88f620cfe182 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:59.564938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15319baa-1a53-419b-9e8d-88f620cfe182 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:59.565652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15319baa-1a53-419b-9e8d-88f620cfe182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:59.568797 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a315b-6a13-44cc-b03f-70e8e3061480 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:59.586962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32602fcb-89a9-4fb5-a722-073a11196375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:59.589849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32602fcb-89a9-4fb5-a722-073a11196375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:59.590225 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32602fcb-89a9-4fb5-a722-073a11196375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:59.592941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32602fcb-89a9-4fb5-a722-073a11196375 admin admin] PATH_INFO: `/v3/users/df5a635afffb4e4aa3592d15c6a5452e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:59.596206 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32602fcb-89a9-4fb5-a722-073a11196375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df5a635afffb4e4aa3592d15c6a5452e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:59.601116 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32602fcb-89a9-4fb5-a722-073a11196375 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:59.602014 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-15319baa-1a53-419b-9e8d-88f620cfe182 None None] MFA Rules not processed for user `6badcf8b92dc43a3bc2342e303dec1da`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:59.604185 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-15319baa-1a53-419b-9e8d-88f620cfe182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:59.710788 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1085/2147] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:59.711679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15319baa-1a53-419b-9e8d-88f620cfe182 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:59.713592 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-15319baa-1a53-419b-9e8d-88f620cfe182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:59.735493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:59.810454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15319baa-1a53-419b-9e8d-88f620cfe182 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:59.815741 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508868e-2ba8-496c-aad0-db131e5287ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:59.816519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508868e-2ba8-496c-aad0-db131e5287ec service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:59.816881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508868e-2ba8-496c-aad0-db131e5287ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:59.817468 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508868e-2ba8-496c-aad0-db131e5287ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:59.819766 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5508868e-2ba8-496c-aad0-db131e5287ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:59.871540 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-32602fcb-89a9-4fb5-a722-073a11196375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df5a635afffb4e4aa3592d15c6a5452e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:59.872461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-32602fcb-89a9-4fb5-a722-073a11196375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df5a635afffb4e4aa3592d15c6a5452e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:59.874915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-32602fcb-89a9-4fb5-a722-073a11196375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df5a635afffb4e4aa3592d15c6a5452e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:59.894229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508868e-2ba8-496c-aad0-db131e5287ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:59.895968 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508868e-2ba8-496c-aad0-db131e5287ec service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:42:59.899196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-32602fcb-89a9-4fb5-a722-073a11196375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df5a635afffb4e4aa3592d15c6a5452e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:59.909032 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1086/2148] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:42:59.912523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2595035-3244-41ce-8e66-169a46974099 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:59.912523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2595035-3244-41ce-8e66-169a46974099 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:59.912523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2595035-3244-41ce-8e66-169a46974099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:59.932837 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-32602fcb-89a9-4fb5-a722-073a11196375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df5a635afffb4e4aa3592d15c6a5452e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:42:59.934142 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a2595035-3244-41ce-8e66-169a46974099 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:42:59.936256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a2595035-3244-41ce-8e66-169a46974099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:59.948582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1087/2149] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:42:59.950667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2595035-3244-41ce-8e66-169a46974099 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:42:59.952024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a2595035-3244-41ce-8e66-169a46974099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:59.965780 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2595035-3244-41ce-8e66-169a46974099 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:42:59.971172 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcf93f-9770-4fad-8ae3-e03dbf254263 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:42:59.972348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcf93f-9770-4fad-8ae3-e03dbf254263 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:42:59.972973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcf93f-9770-4fad-8ae3-e03dbf254263 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:42:59.973296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcf93f-9770-4fad-8ae3-e03dbf254263 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:42:59.975427 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-94fcf93f-9770-4fad-8ae3-e03dbf254263 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:42:59.989806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcf93f-9770-4fad-8ae3-e03dbf254263 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:42:59.991552 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcf93f-9770-4fad-8ae3-e03dbf254263 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:00.008441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1088/2150] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:42:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:00.009620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcf93f-9770-4fad-8ae3-e03dbf254263 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:00.023900 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcf93f-9770-4fad-8ae3-e03dbf254263 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:00.029538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf047b-2b1e-44d7-963f-b664d1ea1f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:00.030499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf047b-2b1e-44d7-963f-b664d1ea1f03 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:00.030796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf047b-2b1e-44d7-963f-b664d1ea1f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:00.031083 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf047b-2b1e-44d7-963f-b664d1ea1f03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:00.032777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf047b-2b1e-44d7-963f-b664d1ea1f03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:43:00.033262 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf047b-2b1e-44d7-963f-b664d1ea1f03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:00.035206 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf047b-2b1e-44d7-963f-b664d1ea1f03 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:00.055718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1089/2152] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:43:00.059119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1063/2152] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:42:59 2022] DELETE /identity/v3/users/df5a635afffb4e4aa3592d15c6a5452e => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:00.066830 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf047b-2b1e-44d7-963f-b664d1ea1f03 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:00.095160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf047b-2b1e-44d7-963f-b664d1ea1f03 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:00.217348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9bb65-cace-48cc-9298-998731120c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:00.221242 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9bb65-cace-48cc-9298-998731120c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:00.221242 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9bb65-cace-48cc-9298-998731120c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:00.222380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9bb65-cace-48cc-9298-998731120c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:00.223244 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9bb65-cace-48cc-9298-998731120c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:00.228028 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9bb65-cace-48cc-9298-998731120c7e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:00.427338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32602fcb-89a9-4fb5-a722-073a11196375 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:00.429438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1090/2153] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:43:00 2022] POST /identity/v3/projects => generated 385 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:43:00.454955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32602fcb-89a9-4fb5-a722-073a11196375 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:00.527454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:00.552164 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c32b9-52da-43be-b539-9f5664ac85b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:00.553007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c32b9-52da-43be-b539-9f5664ac85b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:00.554384 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c32b9-52da-43be-b539-9f5664ac85b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:00.563526 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c32b9-52da-43be-b539-9f5664ac85b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:00.563526 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c32b9-52da-43be-b539-9f5664ac85b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:00.563526 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c32b9-52da-43be-b539-9f5664ac85b7 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:00.705756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9bb65-cace-48cc-9298-998731120c7e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:00.705756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8fe9bb65-cace-48cc-9298-998731120c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:00.722563 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1064/2154] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:43:00 2022] POST /identity/v3/users => generated 293 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:43:00.727210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c32b9-52da-43be-b539-9f5664ac85b7 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:00.737544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9bb65-cace-48cc-9298-998731120c7e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:00.747191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5dac78-6a2c-49f5-b33c-a31834e14efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:00.748509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5dac78-6a2c-49f5-b33c-a31834e14efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:00.749321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5dac78-6a2c-49f5-b33c-a31834e14efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:00.749784 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5dac78-6a2c-49f5-b33c-a31834e14efa admin admin] PATH_INFO: `/v3/projects/3b012572e6ee4654aabe2ec34b35dcc7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:00.752802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5dac78-6a2c-49f5-b33c-a31834e14efa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b012572e6ee4654aabe2ec34b35dcc7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:00.755029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5dac78-6a2c-49f5-b33c-a31834e14efa admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:00.764507 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c32b9-52da-43be-b539-9f5664ac85b7 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:00.773509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ae525-1a0d-4ca0-a2d3-cddcde221b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:00.774357 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ae525-1a0d-4ca0-a2d3-cddcde221b03 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:00.774768 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ae525-1a0d-4ca0-a2d3-cddcde221b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:00.775151 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ae525-1a0d-4ca0-a2d3-cddcde221b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:00.776139 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ae525-1a0d-4ca0-a2d3-cddcde221b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:00.778387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ae525-1a0d-4ca0-a2d3-cddcde221b03 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:00.792590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1065/2155] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:43:00.796778 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ae525-1a0d-4ca0-a2d3-cddcde221b03 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:00.810894 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ae525-1a0d-4ca0-a2d3-cddcde221b03 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:00.825374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74265b31-5371-4e16-9d95-961d9720e15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:00.829504 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74265b31-5371-4e16-9d95-961d9720e15e admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:00.829754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74265b31-5371-4e16-9d95-961d9720e15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:00.830023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74265b31-5371-4e16-9d95-961d9720e15e admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:00.831850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74265b31-5371-4e16-9d95-961d9720e15e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:00.835721 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74265b31-5371-4e16-9d95-961d9720e15e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:00.854287 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1066/2156] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:43:00.858881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74265b31-5371-4e16-9d95-961d9720e15e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:00.880215 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74265b31-5371-4e16-9d95-961d9720e15e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:00.888011 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ef8b7-e245-409a-8c19-c55890949162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:00.888932 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ef8b7-e245-409a-8c19-c55890949162 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:00.889251 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ef8b7-e245-409a-8c19-c55890949162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:00.889532 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ef8b7-e245-409a-8c19-c55890949162 admin admin] PATH_INFO: `/v3/projects/a15ae9d15a64468f80628ed51c2e42d6/users/472049224bbf48699b8a9410ec61aa9e/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:00.909628 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ef8b7-e245-409a-8c19-c55890949162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a15ae9d15a64468f80628ed51c2e42d6, user_id=472049224bbf48699b8a9410ec61aa9e, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:00.916118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ef8b7-e245-409a-8c19-c55890949162 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:01.024150 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1067/2157] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:43:00 2022] PUT /identity/v3/projects/a15ae9d15a64468f80628ed51c2e42d6/users/472049224bbf48699b8a9410ec61aa9e/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:43:01.030077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c9136-0222-4b8e-bb15-dbf8fc65cc3e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:01.030383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c9136-0222-4b8e-bb15-dbf8fc65cc3e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:01.030657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c9136-0222-4b8e-bb15-dbf8fc65cc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:01.047513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5a5dac78-6a2c-49f5-b33c-a31834e14efa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b012572e6ee4654aabe2ec34b35dcc7', 'request_id': 'req-5a5dac78-6a2c-49f5-b33c-a31834e14efa'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:01.063607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5a5dac78-6a2c-49f5-b33c-a31834e14efa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b012572e6ee4654aabe2ec34b35dcc7', 'request_id': 'req-5a5dac78-6a2c-49f5-b33c-a31834e14efa'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:01.076977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6c4c9136-0222-4b8e-bb15-dbf8fc65cc3e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:01.081971 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6c4c9136-0222-4b8e-bb15-dbf8fc65cc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:01.161531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5a5dac78-6a2c-49f5-b33c-a31834e14efa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b012572e6ee4654aabe2ec34b35dcc7', 'request_id': 'req-5a5dac78-6a2c-49f5-b33c-a31834e14efa'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:01.175508 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5a5dac78-6a2c-49f5-b33c-a31834e14efa admin admin] The token cache is being invalidate because project 3b012572e6ee4654aabe2ec34b35dcc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:43:01.175814 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5a5dac78-6a2c-49f5-b33c-a31834e14efa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:01.178364 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5a5dac78-6a2c-49f5-b33c-a31834e14efa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:01.183974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1091/2158] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:43:00 2022] DELETE /identity/v3/projects/3b012572e6ee4654aabe2ec34b35dcc7 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:01.192152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60f54e-f11f-4e22-8cb1-4b8387720917 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:01.192439 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60f54e-f11f-4e22-8cb1-4b8387720917 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:01.192727 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60f54e-f11f-4e22-8cb1-4b8387720917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:01.198766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1068/2159] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:43:01.222089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c9136-0222-4b8e-bb15-dbf8fc65cc3e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:01.223542 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6c4c9136-0222-4b8e-bb15-dbf8fc65cc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:01.243086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9a60f54e-f11f-4e22-8cb1-4b8387720917 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:01.244409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9a60f54e-f11f-4e22-8cb1-4b8387720917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:01.275112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1092/2160] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:01.283058 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473a8de-5f62-47a2-8dbc-429d4cd9c1e0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:01.283347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473a8de-5f62-47a2-8dbc-429d4cd9c1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:01.283642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473a8de-5f62-47a2-8dbc-429d4cd9c1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:01.345096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9473a8de-5f62-47a2-8dbc-429d4cd9c1e0 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:01.345096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9473a8de-5f62-47a2-8dbc-429d4cd9c1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:01.383597 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1093/2161] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:01.384978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473a8de-5f62-47a2-8dbc-429d4cd9c1e0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:01.408458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473a8de-5f62-47a2-8dbc-429d4cd9c1e0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:01.420714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c50dc4-10a7-4b7a-926f-8c28e9c2017c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:01.430248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c50dc4-10a7-4b7a-926f-8c28e9c2017c admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:01.430248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c50dc4-10a7-4b7a-926f-8c28e9c2017c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:01.430248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c50dc4-10a7-4b7a-926f-8c28e9c2017c admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:01.430248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c50dc4-10a7-4b7a-926f-8c28e9c2017c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:43:01.430248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c50dc4-10a7-4b7a-926f-8c28e9c2017c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:01.430248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c50dc4-10a7-4b7a-926f-8c28e9c2017c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:01.445258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c9136-0222-4b8e-bb15-dbf8fc65cc3e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:01.449872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c3f5d-9b78-479d-b42c-83dc3a95af51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:01.450538 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c3f5d-9b78-479d-b42c-83dc3a95af51 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:01.450913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c3f5d-9b78-479d-b42c-83dc3a95af51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:01.451111 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c3f5d-9b78-479d-b42c-83dc3a95af51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:01.456920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1094/2162] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:43:01.459342 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c50dc4-10a7-4b7a-926f-8c28e9c2017c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:01.468145 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c3f5d-9b78-479d-b42c-83dc3a95af51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:01.470770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c3f5d-9b78-479d-b42c-83dc3a95af51 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:01.484486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1069/2163] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:43:01.484829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c50dc4-10a7-4b7a-926f-8c28e9c2017c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:01.509731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d9b25-b074-4337-b694-ddf9c82d64e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:01.510416 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d9b25-b074-4337-b694-ddf9c82d64e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:01.510707 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d9b25-b074-4337-b694-ddf9c82d64e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:01.510985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d9b25-b074-4337-b694-ddf9c82d64e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:01.511689 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d9b25-b074-4337-b694-ddf9c82d64e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:01.514549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d9b25-b074-4337-b694-ddf9c82d64e2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:01.648961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1095/2164] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:43:01 2022] POST /identity/v3/projects => generated 365 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:43:01.653894 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c3f5d-9b78-479d-b42c-83dc3a95af51 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:01.670682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c3f5d-9b78-479d-b42c-83dc3a95af51 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:01.756918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad81494-5648-4c6c-9462-9d7ca47332d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:01.758384 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad81494-5648-4c6c-9462-9d7ca47332d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:01.759075 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad81494-5648-4c6c-9462-9d7ca47332d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:01.759811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad81494-5648-4c6c-9462-9d7ca47332d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:01.760764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad81494-5648-4c6c-9462-9d7ca47332d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:01.763308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad81494-5648-4c6c-9462-9d7ca47332d4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:01.886935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1070/2165] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:43:01 2022] POST /identity/v3/users => generated 283 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:43:01.891362 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d9b25-b074-4337-b694-ddf9c82d64e2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:01.909575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d9b25-b074-4337-b694-ddf9c82d64e2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:01.927037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3a7ba-9db2-4e31-af1d-a45ac16042e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:01.928128 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3a7ba-9db2-4e31-af1d-a45ac16042e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:01.928587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3a7ba-9db2-4e31-af1d-a45ac16042e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:01.929031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3a7ba-9db2-4e31-af1d-a45ac16042e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:01.930016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3a7ba-9db2-4e31-af1d-a45ac16042e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:01.931875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3a7ba-9db2-4e31-af1d-a45ac16042e0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:01.962338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad81494-5648-4c6c-9462-9d7ca47332d4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:01.969990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1096/2166] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:43:01.977944 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad81494-5648-4c6c-9462-9d7ca47332d4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:01.991193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d63fad-92e5-4f23-9a5a-58bb4b9d76dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:01.996595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:01.998050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d63fad-92e5-4f23-9a5a-58bb4b9d76dd admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:01.998573 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d63fad-92e5-4f23-9a5a-58bb4b9d76dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:01.999010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d63fad-92e5-4f23-9a5a-58bb4b9d76dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:02.000141 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d63fad-92e5-4f23-9a5a-58bb4b9d76dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:02.003004 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d63fad-92e5-4f23-9a5a-58bb4b9d76dd admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:02.028653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1071/2167] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:43:02.034447 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3a7ba-9db2-4e31-af1d-a45ac16042e0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:02.048323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3a7ba-9db2-4e31-af1d-a45ac16042e0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:02.053136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fed74-9bb5-4062-b213-7733d0e7364a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:02.059863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fed74-9bb5-4062-b213-7733d0e7364a admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:02.059863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fed74-9bb5-4062-b213-7733d0e7364a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:02.059863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fed74-9bb5-4062-b213-7733d0e7364a admin admin] PATH_INFO: `/v3/projects/747a01002de14df88ced73e2fd16e1a5/users/ce2ff84756fa4126b4b2933db900c4b8/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:02.071893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2fed74-9bb5-4062-b213-7733d0e7364a admin admin] RBAC: Authorizing `identity:create_grant(project_id=747a01002de14df88ced73e2fd16e1a5, user_id=ce2ff84756fa4126b4b2933db900c4b8, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:02.074763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2fed74-9bb5-4062-b213-7733d0e7364a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:02.101167 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:02.166722 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1097/2168] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:43:02 2022] PUT /identity/v3/projects/747a01002de14df88ced73e2fd16e1a5/users/ce2ff84756fa4126b4b2933db900c4b8/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:02.179372 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0b9c7-00dd-429c-b14f-d11750d251a0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:02.179689 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0b9c7-00dd-429c-b14f-d11750d251a0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:02.179956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0b9c7-00dd-429c-b14f-d11750d251a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:02.243601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-35a0b9c7-00dd-429c-b14f-d11750d251a0 None None] MFA Rules not processed for user `ce2ff84756fa4126b4b2933db900c4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:02.245319 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-35a0b9c7-00dd-429c-b14f-d11750d251a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:02.383932 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1072/2169] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:02.385400 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fed74-9bb5-4062-b213-7733d0e7364a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:02.459535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0b9c7-00dd-429c-b14f-d11750d251a0 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:02.461144 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-35a0b9c7-00dd-429c-b14f-d11750d251a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:02.559245 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fed74-9bb5-4062-b213-7733d0e7364a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:02.565425 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d7845-59f0-4844-b1e1-dd2940fb6027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:02.566183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d7845-59f0-4844-b1e1-dd2940fb6027 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:02.569287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d7845-59f0-4844-b1e1-dd2940fb6027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:02.569287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d7845-59f0-4844-b1e1-dd2940fb6027 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:02.569287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d7845-59f0-4844-b1e1-dd2940fb6027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:02.570299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d7845-59f0-4844-b1e1-dd2940fb6027 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:02.585112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0b9c7-00dd-429c-b14f-d11750d251a0 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:02.591283 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012ec52-a9d6-4ab5-afd1-5154849be0af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:02.592160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012ec52-a9d6-4ab5-afd1-5154849be0af service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:02.592683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012ec52-a9d6-4ab5-afd1-5154849be0af service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:02.593155 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012ec52-a9d6-4ab5-afd1-5154849be0af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:02.599186 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c012ec52-a9d6-4ab5-afd1-5154849be0af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:02.700417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012ec52-a9d6-4ab5-afd1-5154849be0af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:02.707937 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012ec52-a9d6-4ab5-afd1-5154849be0af service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:02.724974 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1073/2170] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:02.744201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1098/2171] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:43:02 2022] POST /identity/v3/projects => generated 365 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:43:02.747574 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012ec52-a9d6-4ab5-afd1-5154849be0af service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:02.764884 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012ec52-a9d6-4ab5-afd1-5154849be0af service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:02.844789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b7444-7fc8-4cca-944b-982954392c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:02.846050 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b7444-7fc8-4cca-944b-982954392c99 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:02.846499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b7444-7fc8-4cca-944b-982954392c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:02.847916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b7444-7fc8-4cca-944b-982954392c99 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:02.849859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b7444-7fc8-4cca-944b-982954392c99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:02.853206 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b7444-7fc8-4cca-944b-982954392c99 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:02.977784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1074/2172] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:43:02 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:43:02.979353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d7845-59f0-4844-b1e1-dd2940fb6027 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:02.991616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d7845-59f0-4844-b1e1-dd2940fb6027 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:03.011088 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20001e7-de33-4421-b410-88e8d39967f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:03.011422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20001e7-de33-4421-b410-88e8d39967f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:03.011710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20001e7-de33-4421-b410-88e8d39967f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:03.011989 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20001e7-de33-4421-b410-88e8d39967f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:03.012843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20001e7-de33-4421-b410-88e8d39967f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:03.014808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20001e7-de33-4421-b410-88e8d39967f4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:03.029720 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1099/2173] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:43:03.033089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b7444-7fc8-4cca-944b-982954392c99 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:03.046388 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b7444-7fc8-4cca-944b-982954392c99 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:03.052179 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331829c-a20a-4a42-baf4-e26131dbd546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:03.053109 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331829c-a20a-4a42-baf4-e26131dbd546 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:03.053404 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331829c-a20a-4a42-baf4-e26131dbd546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:03.058543 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331829c-a20a-4a42-baf4-e26131dbd546 admin admin] PATH_INFO: `/v3/projects/3a4ca9a410fc448a8d78861a54f47e4e/users/ea6fe2ab856945e08c2557626a9db95e/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:03.071603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331829c-a20a-4a42-baf4-e26131dbd546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4ca9a410fc448a8d78861a54f47e4e, user_id=ea6fe2ab856945e08c2557626a9db95e, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:03.074574 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331829c-a20a-4a42-baf4-e26131dbd546 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:03.152839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1075/2174] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:43:03 2022] PUT /identity/v3/projects/3a4ca9a410fc448a8d78861a54f47e4e/users/ea6fe2ab856945e08c2557626a9db95e/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:03.156994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20001e7-de33-4421-b410-88e8d39967f4 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:03.170550 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20001e7-de33-4421-b410-88e8d39967f4 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:03.251584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fba52-751f-457f-9a25-4e6752fc0705 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:03.251884 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fba52-751f-457f-9a25-4e6752fc0705 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:03.252161 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fba52-751f-457f-9a25-4e6752fc0705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:03.290058 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a23fba52-751f-457f-9a25-4e6752fc0705 None None] MFA Rules not processed for user `6badcf8b92dc43a3bc2342e303dec1da`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:03.296762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a23fba52-751f-457f-9a25-4e6752fc0705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:03.331198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8b8c1-2163-466c-b5ad-9f9b4ada0e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:03.332899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8b8c1-2163-466c-b5ad-9f9b4ada0e04 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:03.332899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8b8c1-2163-466c-b5ad-9f9b4ada0e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:03.332899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8b8c1-2163-466c-b5ad-9f9b4ada0e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:03.334719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8b8c1-2163-466c-b5ad-9f9b4ada0e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:03.335664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8b8c1-2163-466c-b5ad-9f9b4ada0e04 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:03.354235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1100/2175] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:43:03.355682 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8b8c1-2163-466c-b5ad-9f9b4ada0e04 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:03.369296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8b8c1-2163-466c-b5ad-9f9b4ada0e04 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:03.388806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4441962-d310-4d0b-ac2a-0f561a3f8238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:03.390031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4441962-d310-4d0b-ac2a-0f561a3f8238 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:03.391443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4441962-d310-4d0b-ac2a-0f561a3f8238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:03.391940 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4441962-d310-4d0b-ac2a-0f561a3f8238 admin admin] PATH_INFO: `/v3/projects/3a4ca9a410fc448a8d78861a54f47e4e/users/ea6fe2ab856945e08c2557626a9db95e/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:03.400154 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4441962-d310-4d0b-ac2a-0f561a3f8238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4ca9a410fc448a8d78861a54f47e4e, user_id=ea6fe2ab856945e08c2557626a9db95e, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:03.404023 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4441962-d310-4d0b-ac2a-0f561a3f8238 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:03.436258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1101/2176] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:43:03 2022] PUT /identity/v3/projects/3a4ca9a410fc448a8d78861a54f47e4e/users/ea6fe2ab856945e08c2557626a9db95e/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:43:03.449569 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb0078-f869-4e99-ad14-7044b7b20f65 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:03.452325 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb0078-f869-4e99-ad14-7044b7b20f65 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:03.452878 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb0078-f869-4e99-ad14-7044b7b20f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:03.483035 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1076/2177] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:03.514846 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e0eb0078-f869-4e99-ad14-7044b7b20f65 None None] MFA Rules not processed for user `ea6fe2ab856945e08c2557626a9db95e`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:03.517093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e0eb0078-f869-4e99-ad14-7044b7b20f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:03.668082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1102/2178] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:03.709644 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fba52-751f-457f-9a25-4e6752fc0705 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:03.717712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a23fba52-751f-457f-9a25-4e6752fc0705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:03.821614 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fba52-751f-457f-9a25-4e6752fc0705 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:03.899088 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed407f0f-beb8-415a-a2b1-a455352bf728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:03.900160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed407f0f-beb8-415a-a2b1-a455352bf728 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:03.901294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed407f0f-beb8-415a-a2b1-a455352bf728 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:03.901784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed407f0f-beb8-415a-a2b1-a455352bf728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:03.913750 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed407f0f-beb8-415a-a2b1-a455352bf728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:03.915893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed407f0f-beb8-415a-a2b1-a455352bf728 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:03.931855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1077/2179] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:03.939524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:04.008716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf695d6-7342-4845-bfd6-3d757c1f183b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:04.009082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf695d6-7342-4845-bfd6-3d757c1f183b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:04.009366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf695d6-7342-4845-bfd6-3d757c1f183b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:04.074828 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9cf695d6-7342-4845-bfd6-3d757c1f183b None None] MFA Rules not processed for user `16a831b346e94d79925d017711a07417`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:04.076354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9cf695d6-7342-4845-bfd6-3d757c1f183b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:04.175980 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1103/2180] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:04.190136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed407f0f-beb8-415a-a2b1-a455352bf728 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:04.192308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ed407f0f-beb8-415a-a2b1-a455352bf728 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:04.302359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed407f0f-beb8-415a-a2b1-a455352bf728 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:04.312732 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ab6a6-5c63-4bc9-98c2-1208482a9b7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:04.313866 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ab6a6-5c63-4bc9-98c2-1208482a9b7b service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:04.314157 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ab6a6-5c63-4bc9-98c2-1208482a9b7b service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:04.314442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ab6a6-5c63-4bc9-98c2-1208482a9b7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:04.321957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-759ab6a6-5c63-4bc9-98c2-1208482a9b7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:04.414193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:04.431607 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ab6a6-5c63-4bc9-98c2-1208482a9b7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:04.435907 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ab6a6-5c63-4bc9-98c2-1208482a9b7b service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:04.471865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1078/2181] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:05.299239 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf695d6-7342-4845-bfd6-3d757c1f183b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:05.336172 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:05.411074 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf695d6-7342-4845-bfd6-3d757c1f183b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:05.416922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e5f66-59a0-4065-abf8-d32d303cefc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:05.437286 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e5f66-59a0-4065-abf8-d32d303cefc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:05.441726 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e5f66-59a0-4065-abf8-d32d303cefc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:05.442186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e5f66-59a0-4065-abf8-d32d303cefc5 admin admin] PATH_INFO: `/v3/users/ce2ff84756fa4126b4b2933db900c4b8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:05.444016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e5f66-59a0-4065-abf8-d32d303cefc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce2ff84756fa4126b4b2933db900c4b8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:05.446778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e5f66-59a0-4065-abf8-d32d303cefc5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:05.753617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6f5e5f66-59a0-4065-abf8-d32d303cefc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce2ff84756fa4126b4b2933db900c4b8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:05.771867 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6f5e5f66-59a0-4065-abf8-d32d303cefc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce2ff84756fa4126b4b2933db900c4b8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:05.858995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ab6a6-5c63-4bc9-98c2-1208482a9b7b service nova] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:05.866167 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6f5e5f66-59a0-4065-abf8-d32d303cefc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce2ff84756fa4126b4b2933db900c4b8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:05.866955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6f5e5f66-59a0-4065-abf8-d32d303cefc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce2ff84756fa4126b4b2933db900c4b8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:05.872994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ab6a6-5c63-4bc9-98c2-1208482a9b7b service nova] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:05.876339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6f5e5f66-59a0-4065-abf8-d32d303cefc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce2ff84756fa4126b4b2933db900c4b8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:05.880428 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89e4af-c5c2-4999-a747-ae248af49091 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:05.881378 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89e4af-c5c2-4999-a747-ae248af49091 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:05.882001 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89e4af-c5c2-4999-a747-ae248af49091 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:05.882564 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89e4af-c5c2-4999-a747-ae248af49091 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:05.885820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4a89e4af-c5c2-4999-a747-ae248af49091 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:05.893101 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1104/2182] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:43:05 2022] DELETE /identity/v3/users/ce2ff84756fa4126b4b2933db900c4b8 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:05.895229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3307f9d-2ab7-457c-9456-04260cfa06b3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:05.895670 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3307f9d-2ab7-457c-9456-04260cfa06b3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:05.896051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3307f9d-2ab7-457c-9456-04260cfa06b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:05.933092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e3307f9d-2ab7-457c-9456-04260cfa06b3 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:05.936673 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e3307f9d-2ab7-457c-9456-04260cfa06b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:05.997175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89e4af-c5c2-4999-a747-ae248af49091 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:05.998983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89e4af-c5c2-4999-a747-ae248af49091 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:06.016824 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1079/2183] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:05 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:43:06.045275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1105/2184] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:06.057024 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89e4af-c5c2-4999-a747-ae248af49091 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:06.059165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4a89e4af-c5c2-4999-a747-ae248af49091 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:06.167805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89e4af-c5c2-4999-a747-ae248af49091 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:06.289232 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2e23b-ec78-4992-9cf3-e8ac4b120b11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:06.289566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2e23b-ec78-4992-9cf3-e8ac4b120b11 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:06.289711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2e23b-ec78-4992-9cf3-e8ac4b120b11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:06.289998 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2e23b-ec78-4992-9cf3-e8ac4b120b11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:06.322530 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2e23b-ec78-4992-9cf3-e8ac4b120b11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:06.324168 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2e23b-ec78-4992-9cf3-e8ac4b120b11 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:06.354048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1080/2185] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:06.655089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=15,keystone:SELECT=80,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:07.642345 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3307f9d-2ab7-457c-9456-04260cfa06b3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:07.644387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e3307f9d-2ab7-457c-9456-04260cfa06b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:07.664075 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3307f9d-2ab7-457c-9456-04260cfa06b3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:07.682848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24771158-74f3-45ec-8531-6327a0af65d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:07.684042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24771158-74f3-45ec-8531-6327a0af65d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:07.686329 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24771158-74f3-45ec-8531-6327a0af65d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:07.686866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24771158-74f3-45ec-8531-6327a0af65d4 admin admin] PATH_INFO: `/v3/projects/747a01002de14df88ced73e2fd16e1a5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:07.725466 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24771158-74f3-45ec-8531-6327a0af65d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=747a01002de14df88ced73e2fd16e1a5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:07.727455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24771158-74f3-45ec-8531-6327a0af65d4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:08.029807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-24771158-74f3-45ec-8531-6327a0af65d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '747a01002de14df88ced73e2fd16e1a5', 'request_id': 'req-24771158-74f3-45ec-8531-6327a0af65d4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:08.058509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-24771158-74f3-45ec-8531-6327a0af65d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '747a01002de14df88ced73e2fd16e1a5', 'request_id': 'req-24771158-74f3-45ec-8531-6327a0af65d4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:08.077372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:08.144714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-24771158-74f3-45ec-8531-6327a0af65d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '747a01002de14df88ced73e2fd16e1a5', 'request_id': 'req-24771158-74f3-45ec-8531-6327a0af65d4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:08.160787 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-24771158-74f3-45ec-8531-6327a0af65d4 admin admin] The token cache is being invalidate because project 747a01002de14df88ced73e2fd16e1a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:43:08.161368 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-24771158-74f3-45ec-8531-6327a0af65d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:08.162074 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-24771158-74f3-45ec-8531-6327a0af65d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:08.164578 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1106/2186] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:43:07 2022] DELETE /identity/v3/projects/747a01002de14df88ced73e2fd16e1a5 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:08.168456 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2e23b-ec78-4992-9cf3-e8ac4b120b11 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:08.181242 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2e23b-ec78-4992-9cf3-e8ac4b120b11 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:08.260072 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9bd27-fefd-4bb4-8c69-795a38825c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:08.261013 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9bd27-fefd-4bb4-8c69-795a38825c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:08.261348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9bd27-fefd-4bb4-8c69-795a38825c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:08.261662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9bd27-fefd-4bb4-8c69-795a38825c30 admin admin] PATH_INFO: `/v3/users/ea6fe2ab856945e08c2557626a9db95e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:08.262899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9bd27-fefd-4bb4-8c69-795a38825c30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea6fe2ab856945e08c2557626a9db95e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:08.264965 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9bd27-fefd-4bb4-8c69-795a38825c30 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:08.506783 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-63d9bd27-fefd-4bb4-8c69-795a38825c30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea6fe2ab856945e08c2557626a9db95e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:08.508226 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-63d9bd27-fefd-4bb4-8c69-795a38825c30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea6fe2ab856945e08c2557626a9db95e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:08.511200 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-63d9bd27-fefd-4bb4-8c69-795a38825c30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea6fe2ab856945e08c2557626a9db95e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:08.543182 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-63d9bd27-fefd-4bb4-8c69-795a38825c30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea6fe2ab856945e08c2557626a9db95e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:08.577413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-63d9bd27-fefd-4bb4-8c69-795a38825c30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea6fe2ab856945e08c2557626a9db95e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:08.672817 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1081/2187] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:43:08 2022] DELETE /identity/v3/users/ea6fe2ab856945e08c2557626a9db95e => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:08.931983 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa616f8-605a-409f-bafa-574049e10d13 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:08.932888 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa616f8-605a-409f-bafa-574049e10d13 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:08.933811 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa616f8-605a-409f-bafa-574049e10d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:08.965103 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5fa616f8-605a-409f-bafa-574049e10d13 None None] MFA Rules not processed for user `8b983c0a2a124ac3aa3403e0777612f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:08.966505 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5fa616f8-605a-409f-bafa-574049e10d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:09.080170 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1107/2188] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:09.144995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:09.358079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9bd27-fefd-4bb4-8c69-795a38825c30 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:09.362667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-63d9bd27-fefd-4bb4-8c69-795a38825c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:09.507443 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9bd27-fefd-4bb4-8c69-795a38825c30 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:09.514444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c4898-bc2a-4d74-b204-9c21dba4bdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:09.515354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c4898-bc2a-4d74-b204-9c21dba4bdf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:09.516326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c4898-bc2a-4d74-b204-9c21dba4bdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:09.516644 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c4898-bc2a-4d74-b204-9c21dba4bdf0 admin admin] PATH_INFO: `/v3/projects/3a4ca9a410fc448a8d78861a54f47e4e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:09.517970 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c4898-bc2a-4d74-b204-9c21dba4bdf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a4ca9a410fc448a8d78861a54f47e4e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:09.520355 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c4898-bc2a-4d74-b204-9c21dba4bdf0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:09.620025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:09.679119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-343c4898-bc2a-4d74-b204-9c21dba4bdf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a4ca9a410fc448a8d78861a54f47e4e', 'request_id': 'req-343c4898-bc2a-4d74-b204-9c21dba4bdf0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:09.687595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-343c4898-bc2a-4d74-b204-9c21dba4bdf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a4ca9a410fc448a8d78861a54f47e4e', 'request_id': 'req-343c4898-bc2a-4d74-b204-9c21dba4bdf0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:09.695129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-343c4898-bc2a-4d74-b204-9c21dba4bdf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a4ca9a410fc448a8d78861a54f47e4e', 'request_id': 'req-343c4898-bc2a-4d74-b204-9c21dba4bdf0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:09.819922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-343c4898-bc2a-4d74-b204-9c21dba4bdf0 admin admin] The token cache is being invalidate because project 3a4ca9a410fc448a8d78861a54f47e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:43:09.820219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-343c4898-bc2a-4d74-b204-9c21dba4bdf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:09.821744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-343c4898-bc2a-4d74-b204-9c21dba4bdf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:09.827624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1082/2189] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:43:09 2022] DELETE /identity/v3/projects/3a4ca9a410fc448a8d78861a54f47e4e => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:09.833441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5605f-6fa6-4f08-9753-590c19621792 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:09.834058 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5605f-6fa6-4f08-9753-590c19621792 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:09.834540 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5605f-6fa6-4f08-9753-590c19621792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:09.882862 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6fb5605f-6fa6-4f08-9753-590c19621792 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:09.886808 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6fb5605f-6fa6-4f08-9753-590c19621792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:09.979055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1108/2190] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:09.981757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63120cd5-2b3c-4e2f-a7b5-2930693a6f2a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:09.983954 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63120cd5-2b3c-4e2f-a7b5-2930693a6f2a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:09.983954 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63120cd5-2b3c-4e2f-a7b5-2930693a6f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:10.025453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-63120cd5-2b3c-4e2f-a7b5-2930693a6f2a None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:10.026881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-63120cd5-2b3c-4e2f-a7b5-2930693a6f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:10.041937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5605f-6fa6-4f08-9753-590c19621792 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:10.053149 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1083/2191] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:10.054926 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5605f-6fa6-4f08-9753-590c19621792 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:10.060499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8018d1-6e5d-45b0-a2c1-ca2cefda358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:10.061825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8018d1-6e5d-45b0-a2c1-ca2cefda358d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:10.062313 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8018d1-6e5d-45b0-a2c1-ca2cefda358d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:10.062971 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8018d1-6e5d-45b0-a2c1-ca2cefda358d admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:10.064070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8018d1-6e5d-45b0-a2c1-ca2cefda358d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:43:10.064736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8018d1-6e5d-45b0-a2c1-ca2cefda358d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:10.066623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8018d1-6e5d-45b0-a2c1-ca2cefda358d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:10.086446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63120cd5-2b3c-4e2f-a7b5-2930693a6f2a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:10.091359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1109/2192] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:43:10.097937 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63120cd5-2b3c-4e2f-a7b5-2930693a6f2a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:10.103528 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39417702-d5a7-43ad-862a-7ae1d83a5bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:10.104219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39417702-d5a7-43ad-862a-7ae1d83a5bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:10.104512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39417702-d5a7-43ad-862a-7ae1d83a5bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:10.104804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39417702-d5a7-43ad-862a-7ae1d83a5bc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:10.105601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39417702-d5a7-43ad-862a-7ae1d83a5bc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:10.107590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39417702-d5a7-43ad-862a-7ae1d83a5bc9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:10.199484 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1084/2193] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:43:10 2022] POST /identity/v3/projects => generated 357 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:43:10.204735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8018d1-6e5d-45b0-a2c1-ca2cefda358d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:10.224174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8018d1-6e5d-45b0-a2c1-ca2cefda358d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:10.362564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c39ab-885e-45c7-bb25-37847928cc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:10.363552 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c39ab-885e-45c7-bb25-37847928cc64 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:10.363993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c39ab-885e-45c7-bb25-37847928cc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:10.364545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c39ab-885e-45c7-bb25-37847928cc64 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:10.365470 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c39ab-885e-45c7-bb25-37847928cc64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:10.381416 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c39ab-885e-45c7-bb25-37847928cc64 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:10.470807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d8b0b-563b-41ed-b5ca-76177d61bb18 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:10.471363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d8b0b-563b-41ed-b5ca-76177d61bb18 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:10.471813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d8b0b-563b-41ed-b5ca-76177d61bb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:10.499274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1110/2194] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:43:10 2022] POST /identity/v3/users => generated 279 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:43:10.500189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c39ab-885e-45c7-bb25-37847928cc64 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:10.509214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-697d8b0b-563b-41ed-b5ca-76177d61bb18 None None] MFA Rules not processed for user `472049224bbf48699b8a9410ec61aa9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:10.510443 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-697d8b0b-563b-41ed-b5ca-76177d61bb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:10.513228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c39ab-885e-45c7-bb25-37847928cc64 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:10.519611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05b1af-4867-4085-a923-9c2ad3018205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:10.520386 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05b1af-4867-4085-a923-9c2ad3018205 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:10.520751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05b1af-4867-4085-a923-9c2ad3018205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:10.521155 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05b1af-4867-4085-a923-9c2ad3018205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:10.522096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05b1af-4867-4085-a923-9c2ad3018205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:10.523899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05b1af-4867-4085-a923-9c2ad3018205 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:10.540320 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1111/2195] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:43:10.544528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05b1af-4867-4085-a923-9c2ad3018205 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:10.558488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05b1af-4867-4085-a923-9c2ad3018205 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:10.565025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74575ad6-b2de-4c85-87d2-cc19a69d7046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:10.565946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74575ad6-b2de-4c85-87d2-cc19a69d7046 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:10.566301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74575ad6-b2de-4c85-87d2-cc19a69d7046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:10.572235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74575ad6-b2de-4c85-87d2-cc19a69d7046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:10.572235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74575ad6-b2de-4c85-87d2-cc19a69d7046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:10.572235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74575ad6-b2de-4c85-87d2-cc19a69d7046 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:10.583260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1112/2196] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:43:10.588444 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74575ad6-b2de-4c85-87d2-cc19a69d7046 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:10.607037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1085/2197] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:10.607755 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74575ad6-b2de-4c85-87d2-cc19a69d7046 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:10.613944 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47430c-403e-4bfb-b839-36eb33422a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:10.614858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47430c-403e-4bfb-b839-36eb33422a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:10.615149 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47430c-403e-4bfb-b839-36eb33422a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:10.615436 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47430c-403e-4bfb-b839-36eb33422a4c admin admin] PATH_INFO: `/v3/projects/1b665c15deb043e69049eb0481603765/users/365f708d5f1d44a29b3f8af782931a63/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:10.618758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d8b0b-563b-41ed-b5ca-76177d61bb18 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:10.620911 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-697d8b0b-563b-41ed-b5ca-76177d61bb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:10.639520 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47430c-403e-4bfb-b839-36eb33422a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b665c15deb043e69049eb0481603765, user_id=365f708d5f1d44a29b3f8af782931a63, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:10.644567 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47430c-403e-4bfb-b839-36eb33422a4c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:10.712148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1113/2198] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:43:10 2022] PUT /identity/v3/projects/1b665c15deb043e69049eb0481603765/users/365f708d5f1d44a29b3f8af782931a63/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:10.722118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d8b0b-563b-41ed-b5ca-76177d61bb18 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:10.724878 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060bfe8-2f81-40ea-a4de-5d5948e04146 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:10.725227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060bfe8-2f81-40ea-a4de-5d5948e04146 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:10.725539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060bfe8-2f81-40ea-a4de-5d5948e04146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:10.756682 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6060bfe8-2f81-40ea-a4de-5d5948e04146 None None] MFA Rules not processed for user `365f708d5f1d44a29b3f8af782931a63`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:10.757928 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6060bfe8-2f81-40ea-a4de-5d5948e04146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:10.787786 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d222e2-1f75-4067-98d2-e99e0400b5ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:10.788453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d222e2-1f75-4067-98d2-e99e0400b5ad service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:10.788738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d222e2-1f75-4067-98d2-e99e0400b5ad service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:10.789015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d222e2-1f75-4067-98d2-e99e0400b5ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:10.826079 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1114/2199] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:10 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:10.831673 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060bfe8-2f81-40ea-a4de-5d5948e04146 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:10.872255 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d222e2-1f75-4067-98d2-e99e0400b5ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:10.874160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d222e2-1f75-4067-98d2-e99e0400b5ad service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:10.888474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1086/2200] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:10.962559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060bfe8-2f81-40ea-a4de-5d5948e04146 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:10.971655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee1dc0-9b4a-4649-8298-a3677481d539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:10.972591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee1dc0-9b4a-4649-8298-a3677481d539 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:10.973005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee1dc0-9b4a-4649-8298-a3677481d539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:10.973474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee1dc0-9b4a-4649-8298-a3677481d539 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:10.974714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee1dc0-9b4a-4649-8298-a3677481d539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:10.977283 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee1dc0-9b4a-4649-8298-a3677481d539 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:11.085642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1115/2201] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:43:10 2022] POST /identity/v3/projects => generated 357 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:43:11.091599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d222e2-1f75-4067-98d2-e99e0400b5ad service nova] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:11.103858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d222e2-1f75-4067-98d2-e99e0400b5ad service nova] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:11.191777 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c876c-26bb-42df-b384-04c682f8bac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:11.192774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c876c-26bb-42df-b384-04c682f8bac6 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:11.193648 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c876c-26bb-42df-b384-04c682f8bac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:11.194556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c876c-26bb-42df-b384-04c682f8bac6 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:11.195147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c876c-26bb-42df-b384-04c682f8bac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:11.197590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c876c-26bb-42df-b384-04c682f8bac6 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:11.312317 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1087/2202] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:43:11 2022] POST /identity/v3/users => generated 285 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:43:11.315044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee1dc0-9b4a-4649-8298-a3677481d539 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:11.325385 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee1dc0-9b4a-4649-8298-a3677481d539 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:11.330218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3d7dd-3f55-4c77-97c1-8ed0693379f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:11.331109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3d7dd-3f55-4c77-97c1-8ed0693379f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:11.331229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3d7dd-3f55-4c77-97c1-8ed0693379f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:11.331503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3d7dd-3f55-4c77-97c1-8ed0693379f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:11.332259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3d7dd-3f55-4c77-97c1-8ed0693379f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:11.333932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3d7dd-3f55-4c77-97c1-8ed0693379f7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:11.346207 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1116/2203] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:43:11.350332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c876c-26bb-42df-b384-04c682f8bac6 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:11.361192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c876c-26bb-42df-b384-04c682f8bac6 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:11.368897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3091a7c-8c67-48bc-8e2a-a2ed16fa67fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:11.368897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3091a7c-8c67-48bc-8e2a-a2ed16fa67fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:11.368897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3091a7c-8c67-48bc-8e2a-a2ed16fa67fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:11.368897 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3091a7c-8c67-48bc-8e2a-a2ed16fa67fe admin admin] PATH_INFO: `/v3/projects/7835e375e42b4dd8a1cc131e28baeb36/users/dfb020c7985443dcb5541126c1552219/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:11.387115 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3091a7c-8c67-48bc-8e2a-a2ed16fa67fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7835e375e42b4dd8a1cc131e28baeb36, user_id=dfb020c7985443dcb5541126c1552219, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:11.391037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3091a7c-8c67-48bc-8e2a-a2ed16fa67fe admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:11.455799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:11.498549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1088/2204] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:43:11 2022] PUT /identity/v3/projects/7835e375e42b4dd8a1cc131e28baeb36/users/dfb020c7985443dcb5541126c1552219/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:11.499003 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3d7dd-3f55-4c77-97c1-8ed0693379f7 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:11.519331 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3d7dd-3f55-4c77-97c1-8ed0693379f7 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:11.608349 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:11.663296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e524fc7-ae61-404c-97b7-a3b210789b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:11.664301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e524fc7-ae61-404c-97b7-a3b210789b82 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:11.665543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e524fc7-ae61-404c-97b7-a3b210789b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:11.666158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e524fc7-ae61-404c-97b7-a3b210789b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:11.672503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e524fc7-ae61-404c-97b7-a3b210789b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:11.672503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e524fc7-ae61-404c-97b7-a3b210789b82 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:11.682368 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1117/2205] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:43:11.686720 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3091a7c-8c67-48bc-8e2a-a2ed16fa67fe admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:11.708428 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3091a7c-8c67-48bc-8e2a-a2ed16fa67fe admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:11.713901 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45655c72-95bf-44d4-8927-c26a22eeade4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:11.715208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45655c72-95bf-44d4-8927-c26a22eeade4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:11.715654 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45655c72-95bf-44d4-8927-c26a22eeade4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:11.716084 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45655c72-95bf-44d4-8927-c26a22eeade4 admin admin] PATH_INFO: `/v3/projects/7835e375e42b4dd8a1cc131e28baeb36/users/dfb020c7985443dcb5541126c1552219/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:11.718485 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45655c72-95bf-44d4-8927-c26a22eeade4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7835e375e42b4dd8a1cc131e28baeb36, user_id=dfb020c7985443dcb5541126c1552219, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:11.723606 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45655c72-95bf-44d4-8927-c26a22eeade4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:11.743743 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1089/2206] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:43:11 2022] PUT /identity/v3/projects/7835e375e42b4dd8a1cc131e28baeb36/users/dfb020c7985443dcb5541126c1552219/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:11.745318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f4fb3-2f83-482e-9c74-3bea1b0fe048 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:11.746319 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f4fb3-2f83-482e-9c74-3bea1b0fe048 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:11.747091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f4fb3-2f83-482e-9c74-3bea1b0fe048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:11.780882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-953f4fb3-2f83-482e-9c74-3bea1b0fe048 None None] MFA Rules not processed for user `dfb020c7985443dcb5541126c1552219`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:11.782698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-953f4fb3-2f83-482e-9c74-3bea1b0fe048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:11.890367 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1118/2207] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:11 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:43:11.902072 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45655c72-95bf-44d4-8927-c26a22eeade4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:11.907977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-45655c72-95bf-44d4-8927-c26a22eeade4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:12.011791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45655c72-95bf-44d4-8927-c26a22eeade4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:12.020518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e71cb-2083-4180-82ec-7ce19543150a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:12.021779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e71cb-2083-4180-82ec-7ce19543150a service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:12.022259 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e71cb-2083-4180-82ec-7ce19543150a service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:12.022689 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e71cb-2083-4180-82ec-7ce19543150a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:12.037304 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e71cb-2083-4180-82ec-7ce19543150a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:12.039553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e71cb-2083-4180-82ec-7ce19543150a service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:12.053934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1090/2208] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:11 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:12.377602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f4fb3-2f83-482e-9c74-3bea1b0fe048 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:12.381412 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-953f4fb3-2f83-482e-9c74-3bea1b0fe048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:12.394656 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f4fb3-2f83-482e-9c74-3bea1b0fe048 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:12.398297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d260ff-b707-4395-9e8d-7208cd9304ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:12.398951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d260ff-b707-4395-9e8d-7208cd9304ac service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:12.399248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d260ff-b707-4395-9e8d-7208cd9304ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:12.399530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d260ff-b707-4395-9e8d-7208cd9304ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:12.493174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d260ff-b707-4395-9e8d-7208cd9304ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:12.494933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d260ff-b707-4395-9e8d-7208cd9304ac service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:12.515388 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1119/2209] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:12 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:13.453141 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e71cb-2083-4180-82ec-7ce19543150a service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:13.456054 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2a3e71cb-2083-4180-82ec-7ce19543150a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:13.584207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e71cb-2083-4180-82ec-7ce19543150a service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:13.592463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94286a98-b139-464e-a208-5e8f51ac720b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:13.595343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94286a98-b139-464e-a208-5e8f51ac720b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:13.599752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94286a98-b139-464e-a208-5e8f51ac720b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:13.599752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94286a98-b139-464e-a208-5e8f51ac720b admin admin] PATH_INFO: `/v3/users/961df89ea9fb46f397589baf93ea0335` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:13.613681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94286a98-b139-464e-a208-5e8f51ac720b admin admin] RBAC: Authorizing `identity:delete_user(user_id=961df89ea9fb46f397589baf93ea0335)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:13.617164 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94286a98-b139-464e-a208-5e8f51ac720b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:13.724311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=83,keystone:DELETE=11,keystone:INSERT=6 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:13.837265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-94286a98-b139-464e-a208-5e8f51ac720b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '961df89ea9fb46f397589baf93ea0335'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:13.838067 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-94286a98-b139-464e-a208-5e8f51ac720b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '961df89ea9fb46f397589baf93ea0335'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:13.839029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-94286a98-b139-464e-a208-5e8f51ac720b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '961df89ea9fb46f397589baf93ea0335'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:13.856337 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-94286a98-b139-464e-a208-5e8f51ac720b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '961df89ea9fb46f397589baf93ea0335'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:13.894312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-94286a98-b139-464e-a208-5e8f51ac720b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '961df89ea9fb46f397589baf93ea0335'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:13.999065 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1091/2210] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:43:13 2022] DELETE /identity/v3/users/961df89ea9fb46f397589baf93ea0335 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:14.004127 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79e9cf-1f3e-4e20-8f72-3d89269f113a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:14.004674 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79e9cf-1f3e-4e20-8f72-3d89269f113a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:14.004886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79e9cf-1f3e-4e20-8f72-3d89269f113a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:14.034974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8a79e9cf-1f3e-4e20-8f72-3d89269f113a None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:14.036651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8a79e9cf-1f3e-4e20-8f72-3d89269f113a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:14.126555 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1120/2211] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:14.138154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94286a98-b139-464e-a208-5e8f51ac720b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:14.139604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-94286a98-b139-464e-a208-5e8f51ac720b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:14.234269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94286a98-b139-464e-a208-5e8f51ac720b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:14.239234 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dac2a-1f99-4ba3-afbd-050236fe75ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:14.240426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dac2a-1f99-4ba3-afbd-050236fe75ab service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:14.240724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dac2a-1f99-4ba3-afbd-050236fe75ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:14.241013 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dac2a-1f99-4ba3-afbd-050236fe75ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:14.260349 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3dac2a-1f99-4ba3-afbd-050236fe75ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:14.262117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3dac2a-1f99-4ba3-afbd-050236fe75ab service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:14.275244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1092/2212] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:14.392693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:14.467789 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a89fe-757c-4fdf-8672-6de49f80307f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:14.468085 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a89fe-757c-4fdf-8672-6de49f80307f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:14.468459 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a89fe-757c-4fdf-8672-6de49f80307f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:14.498736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-046a89fe-757c-4fdf-8672-6de49f80307f None None] MFA Rules not processed for user `472049224bbf48699b8a9410ec61aa9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:14.500211 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-046a89fe-757c-4fdf-8672-6de49f80307f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:14.588720 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1121/2213] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:14.994588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dac2a-1f99-4ba3-afbd-050236fe75ab service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:14.996128 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ce3dac2a-1f99-4ba3-afbd-050236fe75ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:15.025562 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dac2a-1f99-4ba3-afbd-050236fe75ab service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:15.060664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f717af-734f-4ec3-82bd-fdea932798e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:15.069530 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f717af-734f-4ec3-82bd-fdea932798e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:15.071005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f717af-734f-4ec3-82bd-fdea932798e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:15.072007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f717af-734f-4ec3-82bd-fdea932798e8 admin admin] PATH_INFO: `/v3/projects/4e0071e2898841c8aceb9086db240a25` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:15.075909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f717af-734f-4ec3-82bd-fdea932798e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e0071e2898841c8aceb9086db240a25)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:15.080686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f717af-734f-4ec3-82bd-fdea932798e8 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:15.241632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b6f717af-734f-4ec3-82bd-fdea932798e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e0071e2898841c8aceb9086db240a25', 'request_id': 'req-b6f717af-734f-4ec3-82bd-fdea932798e8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:15.253606 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b6f717af-734f-4ec3-82bd-fdea932798e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e0071e2898841c8aceb9086db240a25', 'request_id': 'req-b6f717af-734f-4ec3-82bd-fdea932798e8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:15.264023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b6f717af-734f-4ec3-82bd-fdea932798e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e0071e2898841c8aceb9086db240a25', 'request_id': 'req-b6f717af-734f-4ec3-82bd-fdea932798e8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:15.352389 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b6f717af-734f-4ec3-82bd-fdea932798e8 admin admin] The token cache is being invalidate because project 4e0071e2898841c8aceb9086db240a25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:43:15.352815 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b6f717af-734f-4ec3-82bd-fdea932798e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:15.353621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-b6f717af-734f-4ec3-82bd-fdea932798e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:15.361276 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1093/2214] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:43:14 2022] DELETE /identity/v3/projects/4e0071e2898841c8aceb9086db240a25 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:15.361432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a89fe-757c-4fdf-8672-6de49f80307f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:15.373217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a89fe-757c-4fdf-8672-6de49f80307f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:15.445462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742bfc7b-4784-4748-810e-ed7d7803a4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:15.446365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bfc7b-4784-4748-810e-ed7d7803a4bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:15.446654 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bfc7b-4784-4748-810e-ed7d7803a4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:15.446930 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bfc7b-4784-4748-810e-ed7d7803a4bd admin admin] PATH_INFO: `/v3/users/28553b6adc6c42c29a1bec131a44968b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:15.448432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742bfc7b-4784-4748-810e-ed7d7803a4bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=28553b6adc6c42c29a1bec131a44968b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:15.450485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742bfc7b-4784-4748-810e-ed7d7803a4bd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:15.706274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-742bfc7b-4784-4748-810e-ed7d7803a4bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28553b6adc6c42c29a1bec131a44968b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:15.727876 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-742bfc7b-4784-4748-810e-ed7d7803a4bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28553b6adc6c42c29a1bec131a44968b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:15.808119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-742bfc7b-4784-4748-810e-ed7d7803a4bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28553b6adc6c42c29a1bec131a44968b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:15.809694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-742bfc7b-4784-4748-810e-ed7d7803a4bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28553b6adc6c42c29a1bec131a44968b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:15.819209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-742bfc7b-4784-4748-810e-ed7d7803a4bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28553b6adc6c42c29a1bec131a44968b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:15.829755 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1122/2215] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:43:15 2022] DELETE /identity/v3/users/28553b6adc6c42c29a1bec131a44968b => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:43:16.518783 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:16.657614 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f717af-734f-4ec3-82bd-fdea932798e8 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:16.659326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b6f717af-734f-4ec3-82bd-fdea932798e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:16.680283 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f717af-734f-4ec3-82bd-fdea932798e8 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:16.754885 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928f0a2-1b6a-40fd-a502-27cad11a405c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:16.756267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928f0a2-1b6a-40fd-a502-27cad11a405c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:16.756755 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928f0a2-1b6a-40fd-a502-27cad11a405c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:16.757419 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928f0a2-1b6a-40fd-a502-27cad11a405c admin admin] PATH_INFO: `/v3/projects/b42e249de8fb4db3be4f7777d9dbb00c` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:16.761836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928f0a2-1b6a-40fd-a502-27cad11a405c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b42e249de8fb4db3be4f7777d9dbb00c)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:16.765308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928f0a2-1b6a-40fd-a502-27cad11a405c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:16.925296 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9928f0a2-1b6a-40fd-a502-27cad11a405c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b42e249de8fb4db3be4f7777d9dbb00c', 'request_id': 'req-9928f0a2-1b6a-40fd-a502-27cad11a405c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:16.932705 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9928f0a2-1b6a-40fd-a502-27cad11a405c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b42e249de8fb4db3be4f7777d9dbb00c', 'request_id': 'req-9928f0a2-1b6a-40fd-a502-27cad11a405c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:16.946503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9928f0a2-1b6a-40fd-a502-27cad11a405c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b42e249de8fb4db3be4f7777d9dbb00c', 'request_id': 'req-9928f0a2-1b6a-40fd-a502-27cad11a405c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:16.998054 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9928f0a2-1b6a-40fd-a502-27cad11a405c admin admin] The token cache is being invalidate because project b42e249de8fb4db3be4f7777d9dbb00c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:43:16.998489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9928f0a2-1b6a-40fd-a502-27cad11a405c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:16.999220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9928f0a2-1b6a-40fd-a502-27cad11a405c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:17.005133 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1094/2216] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:43:16 2022] DELETE /identity/v3/projects/b42e249de8fb4db3be4f7777d9dbb00c => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:17.006588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742bfc7b-4784-4748-810e-ed7d7803a4bd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:17.009508 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-742bfc7b-4784-4748-810e-ed7d7803a4bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:17.028616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742bfc7b-4784-4748-810e-ed7d7803a4bd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:17.111021 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4545a5-f6f3-4230-86a5-fcdf9960b088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:17.112428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4545a5-f6f3-4230-86a5-fcdf9960b088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:17.112962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4545a5-f6f3-4230-86a5-fcdf9960b088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:17.113428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4545a5-f6f3-4230-86a5-fcdf9960b088 admin admin] PATH_INFO: `/v3/users/e2a9a8df58bf4856a50387bf50da623d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:17.119611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4545a5-f6f3-4230-86a5-fcdf9960b088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2a9a8df58bf4856a50387bf50da623d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:17.119611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4545a5-f6f3-4230-86a5-fcdf9960b088 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:17.236222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:17.370258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-af4545a5-f6f3-4230-86a5-fcdf9960b088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2a9a8df58bf4856a50387bf50da623d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:17.382546 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-af4545a5-f6f3-4230-86a5-fcdf9960b088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2a9a8df58bf4856a50387bf50da623d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:17.528667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-af4545a5-f6f3-4230-86a5-fcdf9960b088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2a9a8df58bf4856a50387bf50da623d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:17.529332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-af4545a5-f6f3-4230-86a5-fcdf9960b088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2a9a8df58bf4856a50387bf50da623d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:17.538205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-af4545a5-f6f3-4230-86a5-fcdf9960b088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2a9a8df58bf4856a50387bf50da623d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:17.546491 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1123/2217] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:43:17 2022] DELETE /identity/v3/users/e2a9a8df58bf4856a50387bf50da623d => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:43:17.991163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928f0a2-1b6a-40fd-a502-27cad11a405c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:17.992644 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9928f0a2-1b6a-40fd-a502-27cad11a405c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:18.006228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928f0a2-1b6a-40fd-a502-27cad11a405c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:18.064705 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f18898-0f5d-4f7e-8f6f-ce6f824a6f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:18.065993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f18898-0f5d-4f7e-8f6f-ce6f824a6f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:18.066283 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f18898-0f5d-4f7e-8f6f-ce6f824a6f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:18.066565 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f18898-0f5d-4f7e-8f6f-ce6f824a6f45 admin admin] PATH_INFO: `/v3/projects/0edf674f1f16448b895cd354efdc6406` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:18.067799 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f18898-0f5d-4f7e-8f6f-ce6f824a6f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0edf674f1f16448b895cd354efdc6406)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:18.069781 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f18898-0f5d-4f7e-8f6f-ce6f824a6f45 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:18.173925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-74f18898-0f5d-4f7e-8f6f-ce6f824a6f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0edf674f1f16448b895cd354efdc6406', 'request_id': 'req-74f18898-0f5d-4f7e-8f6f-ce6f824a6f45'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:18.179172 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-74f18898-0f5d-4f7e-8f6f-ce6f824a6f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0edf674f1f16448b895cd354efdc6406', 'request_id': 'req-74f18898-0f5d-4f7e-8f6f-ce6f824a6f45'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:18.185250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-74f18898-0f5d-4f7e-8f6f-ce6f824a6f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0edf674f1f16448b895cd354efdc6406', 'request_id': 'req-74f18898-0f5d-4f7e-8f6f-ce6f824a6f45'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:18.247553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:18.292690 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-74f18898-0f5d-4f7e-8f6f-ce6f824a6f45 admin admin] The token cache is being invalidate because project 0edf674f1f16448b895cd354efdc6406 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:43:18.292987 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-74f18898-0f5d-4f7e-8f6f-ce6f824a6f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:18.293654 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-74f18898-0f5d-4f7e-8f6f-ce6f824a6f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:18.295915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1095/2218] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:43:17 2022] DELETE /identity/v3/projects/0edf674f1f16448b895cd354efdc6406 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:18.302374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89184e1-fb97-4194-8d5f-3aa1a8e2f0ad None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:18.302824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89184e1-fb97-4194-8d5f-3aa1a8e2f0ad None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:18.303256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89184e1-fb97-4194-8d5f-3aa1a8e2f0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:18.323157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d89184e1-fb97-4194-8d5f-3aa1a8e2f0ad None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:18.324467 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d89184e1-fb97-4194-8d5f-3aa1a8e2f0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:18.386142 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1124/2219] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:18.394278 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2bc41-80a0-4026-a350-fd95b94f8e1d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:18.394485 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2bc41-80a0-4026-a350-fd95b94f8e1d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:18.394779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2bc41-80a0-4026-a350-fd95b94f8e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:18.414114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-30f2bc41-80a0-4026-a350-fd95b94f8e1d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:18.415386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-30f2bc41-80a0-4026-a350-fd95b94f8e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:18.424580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1096/2220] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:18.428849 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89184e1-fb97-4194-8d5f-3aa1a8e2f0ad None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:18.438681 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89184e1-fb97-4194-8d5f-3aa1a8e2f0ad None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:18.444645 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee30d4-3fe2-456b-be34-34455638d3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:18.445346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee30d4-3fe2-456b-be34-34455638d3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:18.445636 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee30d4-3fe2-456b-be34-34455638d3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:18.445919 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee30d4-3fe2-456b-be34-34455638d3f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:18.446658 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee30d4-3fe2-456b-be34-34455638d3f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:43:18.446964 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee30d4-3fe2-456b-be34-34455638d3f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:18.448568 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee30d4-3fe2-456b-be34-34455638d3f5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:18.460609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1125/2221] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:43:18.472222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2bc41-80a0-4026-a350-fd95b94f8e1d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:18.481290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2bc41-80a0-4026-a350-fd95b94f8e1d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:18.487031 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6a054-5e06-4684-94f5-ed21beeffef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:18.487686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6a054-5e06-4684-94f5-ed21beeffef5 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:18.487970 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6a054-5e06-4684-94f5-ed21beeffef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:18.488246 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6a054-5e06-4684-94f5-ed21beeffef5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:18.488939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6a054-5e06-4684-94f5-ed21beeffef5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:18.490876 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6a054-5e06-4684-94f5-ed21beeffef5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:18.629238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1097/2222] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:43:18 2022] POST /identity/v3/projects => generated 391 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:43:18.630515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee30d4-3fe2-456b-be34-34455638d3f5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:18.640919 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee30d4-3fe2-456b-be34-34455638d3f5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:18.702130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75724b6f-061e-4531-bc9b-fe1c78f37902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:18.702846 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75724b6f-061e-4531-bc9b-fe1c78f37902 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:18.703142 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75724b6f-061e-4531-bc9b-fe1c78f37902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:18.703423 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75724b6f-061e-4531-bc9b-fe1c78f37902 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:18.704118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75724b6f-061e-4531-bc9b-fe1c78f37902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:18.706197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75724b6f-061e-4531-bc9b-fe1c78f37902 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:18.733137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:18.791190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1126/2223] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:43:18 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:43:18.795500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6a054-5e06-4684-94f5-ed21beeffef5 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:18.806487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6a054-5e06-4684-94f5-ed21beeffef5 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:18.811116 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7e842-1e05-4634-99bc-718e1ea1f119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:18.811773 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7e842-1e05-4634-99bc-718e1ea1f119 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:18.812061 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7e842-1e05-4634-99bc-718e1ea1f119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:18.812348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7e842-1e05-4634-99bc-718e1ea1f119 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:18.813077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7e842-1e05-4634-99bc-718e1ea1f119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:18.821368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7e842-1e05-4634-99bc-718e1ea1f119 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:18.825396 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1098/2224] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:43:18.829225 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75724b6f-061e-4531-bc9b-fe1c78f37902 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:18.839193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75724b6f-061e-4531-bc9b-fe1c78f37902 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:18.843825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85956dbe-5245-46b9-b836-f26f4d6041bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:18.844475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85956dbe-5245-46b9-b836-f26f4d6041bd admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:18.844760 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85956dbe-5245-46b9-b836-f26f4d6041bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:18.845630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85956dbe-5245-46b9-b836-f26f4d6041bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:18.846535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85956dbe-5245-46b9-b836-f26f4d6041bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:18.848120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85956dbe-5245-46b9-b836-f26f4d6041bd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:18.857859 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1127/2225] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:43:18.865303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7e842-1e05-4634-99bc-718e1ea1f119 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:18.875045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7e842-1e05-4634-99bc-718e1ea1f119 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:18.879770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc10d01-053a-428e-a207-9b8eacc34063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:18.880684 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc10d01-053a-428e-a207-9b8eacc34063 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:18.880968 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc10d01-053a-428e-a207-9b8eacc34063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:18.881274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc10d01-053a-428e-a207-9b8eacc34063 admin admin] PATH_INFO: `/v3/projects/ccc77f8e62cd4067bda713897b22e9ff/users/b76a9ea6df044db180be8708f3ccdf46/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:18.895446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc10d01-053a-428e-a207-9b8eacc34063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccc77f8e62cd4067bda713897b22e9ff, user_id=b76a9ea6df044db180be8708f3ccdf46, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:18.898220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc10d01-053a-428e-a207-9b8eacc34063 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:18.968231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1099/2226] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:43:18 2022] PUT /identity/v3/projects/ccc77f8e62cd4067bda713897b22e9ff/users/b76a9ea6df044db180be8708f3ccdf46/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:18.981246 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce975f-f88d-4b58-9c36-4515f9332e7e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:18.981246 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce975f-f88d-4b58-9c36-4515f9332e7e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:18.981246 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce975f-f88d-4b58-9c36-4515f9332e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:19.013294 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c9ce975f-f88d-4b58-9c36-4515f9332e7e None None] MFA Rules not processed for user `b76a9ea6df044db180be8708f3ccdf46`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:19.014535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c9ce975f-f88d-4b58-9c36-4515f9332e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:19.089677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1128/2227] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:19.101219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc10d01-053a-428e-a207-9b8eacc34063 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:19.102961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-dcc10d01-053a-428e-a207-9b8eacc34063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:19.165730 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc10d01-053a-428e-a207-9b8eacc34063 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:19.169437 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1f567-cb42-469a-a72e-7e707c06ece5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:19.170353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1f567-cb42-469a-a72e-7e707c06ece5 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:19.170644 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1f567-cb42-469a-a72e-7e707c06ece5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:19.170927 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1f567-cb42-469a-a72e-7e707c06ece5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:19.179446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1f567-cb42-469a-a72e-7e707c06ece5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:19.180896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1f567-cb42-469a-a72e-7e707c06ece5 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:19.189780 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1100/2228] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:20.738780 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:20.887969 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:22.572852 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:22.664488 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:23.259100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:23.487389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:23.629502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:23.772397 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:24.224573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:24.276460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:24.789643 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:24.818158 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:24.961945 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:24.990307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:25.013151 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:25.093130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:25.173449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:25.343975 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:25.885935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:26.206724 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:26.362680 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:27.195711 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:27.276237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:27.538436 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:27.606956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:28.351319 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:28.521946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:28.876199 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:28.899034 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:28.990526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:29.076005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:29.128489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:29.201865 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:36.384795 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce975f-f88d-4b58-9c36-4515f9332e7e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:36.386415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c9ce975f-f88d-4b58-9c36-4515f9332e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:36.477017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce975f-f88d-4b58-9c36-4515f9332e7e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:36.483526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe5f70-36b1-422e-a23a-fe4f338fbb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:36.484534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe5f70-36b1-422e-a23a-fe4f338fbb53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:36.484932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe5f70-36b1-422e-a23a-fe4f338fbb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:36.485537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe5f70-36b1-422e-a23a-fe4f338fbb53 admin admin] PATH_INFO: `/v3/users/0f9eb51be45749ba867402297051014f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:36.487812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe5f70-36b1-422e-a23a-fe4f338fbb53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f9eb51be45749ba867402297051014f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:36.492041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe5f70-36b1-422e-a23a-fe4f338fbb53 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:36.723467 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-01fe5f70-36b1-422e-a23a-fe4f338fbb53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f9eb51be45749ba867402297051014f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:36.740780 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-01fe5f70-36b1-422e-a23a-fe4f338fbb53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f9eb51be45749ba867402297051014f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:36.802313 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-01fe5f70-36b1-422e-a23a-fe4f338fbb53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f9eb51be45749ba867402297051014f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:36.802929 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-01fe5f70-36b1-422e-a23a-fe4f338fbb53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f9eb51be45749ba867402297051014f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:36.812115 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-01fe5f70-36b1-422e-a23a-fe4f338fbb53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f9eb51be45749ba867402297051014f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:36.819452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1129/2229] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:43:36 2022] DELETE /identity/v3/users/0f9eb51be45749ba867402297051014f => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:36.824085 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be901fb-7b48-4b4e-8dff-98b8fafd556b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:36.828155 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be901fb-7b48-4b4e-8dff-98b8fafd556b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:36.828155 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be901fb-7b48-4b4e-8dff-98b8fafd556b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:36.852119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6be901fb-7b48-4b4e-8dff-98b8fafd556b None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:36.853541 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6be901fb-7b48-4b4e-8dff-98b8fafd556b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:36.934339 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1101/2230] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:36.944956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe5f70-36b1-422e-a23a-fe4f338fbb53 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:36.946648 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-01fe5f70-36b1-422e-a23a-fe4f338fbb53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:36.971920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:37.010628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe5f70-36b1-422e-a23a-fe4f338fbb53 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:37.015265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcafb81-ec6f-4167-bac7-0987385d5522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:37.015911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcafb81-ec6f-4167-bac7-0987385d5522 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:37.016195 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcafb81-ec6f-4167-bac7-0987385d5522 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:37.016469 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcafb81-ec6f-4167-bac7-0987385d5522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:37.026645 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcafb81-ec6f-4167-bac7-0987385d5522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:37.028240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcafb81-ec6f-4167-bac7-0987385d5522 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:37.038660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1130/2231] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:37.515552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be901fb-7b48-4b4e-8dff-98b8fafd556b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:37.518045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6be901fb-7b48-4b4e-8dff-98b8fafd556b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:37.531079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be901fb-7b48-4b4e-8dff-98b8fafd556b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:37.536696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809331a4-fb32-4fa4-a9d9-6234b282d3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:37.538038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809331a4-fb32-4fa4-a9d9-6234b282d3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:37.538884 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809331a4-fb32-4fa4-a9d9-6234b282d3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:37.539175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809331a4-fb32-4fa4-a9d9-6234b282d3e8 admin admin] PATH_INFO: `/v3/projects/bb1b995c217142cc9bd40c90e2599888` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:37.540390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809331a4-fb32-4fa4-a9d9-6234b282d3e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb1b995c217142cc9bd40c90e2599888)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:37.543795 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809331a4-fb32-4fa4-a9d9-6234b282d3e8 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:37.743821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:37.756085 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-809331a4-fb32-4fa4-a9d9-6234b282d3e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb1b995c217142cc9bd40c90e2599888', 'request_id': 'req-809331a4-fb32-4fa4-a9d9-6234b282d3e8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:37.761553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-809331a4-fb32-4fa4-a9d9-6234b282d3e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb1b995c217142cc9bd40c90e2599888', 'request_id': 'req-809331a4-fb32-4fa4-a9d9-6234b282d3e8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:37.769152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-809331a4-fb32-4fa4-a9d9-6234b282d3e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb1b995c217142cc9bd40c90e2599888', 'request_id': 'req-809331a4-fb32-4fa4-a9d9-6234b282d3e8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:37.857639 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-809331a4-fb32-4fa4-a9d9-6234b282d3e8 admin admin] The token cache is being invalidate because project bb1b995c217142cc9bd40c90e2599888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:43:37.858550 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-809331a4-fb32-4fa4-a9d9-6234b282d3e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:37.859245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-809331a4-fb32-4fa4-a9d9-6234b282d3e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:37.862108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1102/2232] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:43:37 2022] DELETE /identity/v3/projects/bb1b995c217142cc9bd40c90e2599888 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:37.867143 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d53b37-9af0-4c46-bca4-e76f00cc48b0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:37.867594 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d53b37-9af0-4c46-bca4-e76f00cc48b0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:37.867777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d53b37-9af0-4c46-bca4-e76f00cc48b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:37.890373 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-50d53b37-9af0-4c46-bca4-e76f00cc48b0 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:37.900164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-50d53b37-9af0-4c46-bca4-e76f00cc48b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:37.968571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1131/2233] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:37.983651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730ead3-1004-4791-9f50-8b8f632ff7b1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:37.983651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730ead3-1004-4791-9f50-8b8f632ff7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:37.983651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730ead3-1004-4791-9f50-8b8f632ff7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:37.999342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-4730ead3-1004-4791-9f50-8b8f632ff7b1 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:38.001618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4730ead3-1004-4791-9f50-8b8f632ff7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:38.010240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1103/2234] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:38.014701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d53b37-9af0-4c46-bca4-e76f00cc48b0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:38.025855 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d53b37-9af0-4c46-bca4-e76f00cc48b0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:38.030829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1edec-4023-4507-a691-5e0c909e2600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:38.031501 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1edec-4023-4507-a691-5e0c909e2600 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:38.031786 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1edec-4023-4507-a691-5e0c909e2600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:38.032065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1edec-4023-4507-a691-5e0c909e2600 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:38.032826 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1edec-4023-4507-a691-5e0c909e2600 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:43:38.033175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1edec-4023-4507-a691-5e0c909e2600 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:38.036063 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1edec-4023-4507-a691-5e0c909e2600 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:38.052800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1132/2235] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:43:38.056867 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730ead3-1004-4791-9f50-8b8f632ff7b1 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:38.069242 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730ead3-1004-4791-9f50-8b8f632ff7b1 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:38.079467 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb61f3a-0e01-4524-bc72-e12d4cd8572a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:38.080260 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb61f3a-0e01-4524-bc72-e12d4cd8572a admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:38.080696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb61f3a-0e01-4524-bc72-e12d4cd8572a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:38.082398 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb61f3a-0e01-4524-bc72-e12d4cd8572a admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:38.083232 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb61f3a-0e01-4524-bc72-e12d4cd8572a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:38.085370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb61f3a-0e01-4524-bc72-e12d4cd8572a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:38.243531 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1104/2236] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:43:38 2022] POST /identity/v3/projects => generated 381 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:43:38.247545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1edec-4023-4507-a691-5e0c909e2600 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:38.261542 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1edec-4023-4507-a691-5e0c909e2600 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:38.329786 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08940b-006a-4463-88cf-e838ab327c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:38.330387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08940b-006a-4463-88cf-e838ab327c37 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:38.331374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08940b-006a-4463-88cf-e838ab327c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:38.331756 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08940b-006a-4463-88cf-e838ab327c37 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:38.332612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb08940b-006a-4463-88cf-e838ab327c37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:38.334588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb08940b-006a-4463-88cf-e838ab327c37 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:38.479733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1133/2237] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:43:38 2022] POST /identity/v3/users => generated 291 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:43:38.481220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb61f3a-0e01-4524-bc72-e12d4cd8572a admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:38.492438 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb61f3a-0e01-4524-bc72-e12d4cd8572a admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:38.497448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d976c-df36-4da1-81da-af05754a14ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:38.498114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d976c-df36-4da1-81da-af05754a14ef admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:38.498404 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d976c-df36-4da1-81da-af05754a14ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:38.498678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d976c-df36-4da1-81da-af05754a14ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:38.499462 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d976c-df36-4da1-81da-af05754a14ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:38.501075 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d976c-df36-4da1-81da-af05754a14ef admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:38.514627 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1105/2238] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:43:38.521286 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08940b-006a-4463-88cf-e838ab327c37 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:38.536904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08940b-006a-4463-88cf-e838ab327c37 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:38.543803 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99597d2d-4098-49c6-a455-b0e8d40abb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:38.544768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99597d2d-4098-49c6-a455-b0e8d40abb18 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:38.545225 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99597d2d-4098-49c6-a455-b0e8d40abb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:38.545904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99597d2d-4098-49c6-a455-b0e8d40abb18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:38.547111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99597d2d-4098-49c6-a455-b0e8d40abb18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:38.550793 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99597d2d-4098-49c6-a455-b0e8d40abb18 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:38.564537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1134/2239] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:43:38.569171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d976c-df36-4da1-81da-af05754a14ef admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:38.579524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d976c-df36-4da1-81da-af05754a14ef admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:38.588383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdeff9-78f8-4023-97c3-1b0a3998a9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:38.588383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdeff9-78f8-4023-97c3-1b0a3998a9c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:38.588383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdeff9-78f8-4023-97c3-1b0a3998a9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:38.588383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdeff9-78f8-4023-97c3-1b0a3998a9c0 admin admin] PATH_INFO: `/v3/projects/3fbffa4eae4b4ff89a074930f4e9ab30/users/b60d58f3ec924d219aed4b7ab6dda03a/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:38.604079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdeff9-78f8-4023-97c3-1b0a3998a9c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fbffa4eae4b4ff89a074930f4e9ab30, user_id=b60d58f3ec924d219aed4b7ab6dda03a, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:38.607059 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdeff9-78f8-4023-97c3-1b0a3998a9c0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:38.710774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1106/2240] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:43:38 2022] PUT /identity/v3/projects/3fbffa4eae4b4ff89a074930f4e9ab30/users/b60d58f3ec924d219aed4b7ab6dda03a/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:38.721117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61a28d-dfda-430b-8766-18c4468db222 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:38.721460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61a28d-dfda-430b-8766-18c4468db222 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:38.721754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61a28d-dfda-430b-8766-18c4468db222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:38.749598 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8a61a28d-dfda-430b-8766-18c4468db222 None None] MFA Rules not processed for user `b60d58f3ec924d219aed4b7ab6dda03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:38.750855 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8a61a28d-dfda-430b-8766-18c4468db222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:38.774707 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:38.828222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1135/2241] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:38.840162 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdeff9-78f8-4023-97c3-1b0a3998a9c0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:38.842414 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3bcdeff9-78f8-4023-97c3-1b0a3998a9c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:38.949709 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdeff9-78f8-4023-97c3-1b0a3998a9c0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:38.964071 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc280a6f-0b47-4bdb-81b8-455ebeb93825 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:38.964495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc280a6f-0b47-4bdb-81b8-455ebeb93825 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:38.964792 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc280a6f-0b47-4bdb-81b8-455ebeb93825 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:38.965291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc280a6f-0b47-4bdb-81b8-455ebeb93825 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:38.976356 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc280a6f-0b47-4bdb-81b8-455ebeb93825 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:38.977907 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc280a6f-0b47-4bdb-81b8-455ebeb93825 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:38.989214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1107/2242] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:41.051107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f8eac-8b72-4af7-8c00-53500a495004 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:41.051701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f8eac-8b72-4af7-8c00-53500a495004 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:41.051798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f8eac-8b72-4af7-8c00-53500a495004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:41.070060 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-be2f8eac-8b72-4af7-8c00-53500a495004 None None] MFA Rules not processed for user `6badcf8b92dc43a3bc2342e303dec1da`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:41.071269 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-be2f8eac-8b72-4af7-8c00-53500a495004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:41.125132 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1136/2243] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:41.840316 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc280a6f-0b47-4bdb-81b8-455ebeb93825 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:41.841978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fc280a6f-0b47-4bdb-81b8-455ebeb93825 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:41.897248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:41.927562 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc280a6f-0b47-4bdb-81b8-455ebeb93825 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:41.934087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a8b29-057d-486c-8b36-813d30f495c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:41.935177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a8b29-057d-486c-8b36-813d30f495c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:41.935470 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a8b29-057d-486c-8b36-813d30f495c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:41.935750 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a8b29-057d-486c-8b36-813d30f495c6 admin admin] PATH_INFO: `/v3/users/b76a9ea6df044db180be8708f3ccdf46` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:41.937157 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a8b29-057d-486c-8b36-813d30f495c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b76a9ea6df044db180be8708f3ccdf46)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:41.939332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a8b29-057d-486c-8b36-813d30f495c6 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:42.025080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f8eac-8b72-4af7-8c00-53500a495004 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:42.035941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f8eac-8b72-4af7-8c00-53500a495004 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:42.041564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da665567-431a-419b-a454-e7b32514bc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:42.042569 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da665567-431a-419b-a454-e7b32514bc0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:42.042865 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da665567-431a-419b-a454-e7b32514bc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:42.043153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da665567-431a-419b-a454-e7b32514bc0d admin admin] PATH_INFO: `/v3/users/b60d58f3ec924d219aed4b7ab6dda03a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:42.044399 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da665567-431a-419b-a454-e7b32514bc0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b60d58f3ec924d219aed4b7ab6dda03a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:42.046881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da665567-431a-419b-a454-e7b32514bc0d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:42.141203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ab5a8b29-057d-486c-8b36-813d30f495c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b76a9ea6df044db180be8708f3ccdf46'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:42.141815 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ab5a8b29-057d-486c-8b36-813d30f495c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b76a9ea6df044db180be8708f3ccdf46'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:42.142426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ab5a8b29-057d-486c-8b36-813d30f495c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b76a9ea6df044db180be8708f3ccdf46'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:42.149950 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ab5a8b29-057d-486c-8b36-813d30f495c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b76a9ea6df044db180be8708f3ccdf46'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:42.162385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ab5a8b29-057d-486c-8b36-813d30f495c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b76a9ea6df044db180be8708f3ccdf46'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:42.228172 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-da665567-431a-419b-a454-e7b32514bc0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b60d58f3ec924d219aed4b7ab6dda03a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:42.243063 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-da665567-431a-419b-a454-e7b32514bc0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b60d58f3ec924d219aed4b7ab6dda03a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:42.260494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1108/2244] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:43:41 2022] DELETE /identity/v3/users/b76a9ea6df044db180be8708f3ccdf46 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:42.264798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23129e24-fabd-4dfb-81ac-1e93064a230f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:42.265121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23129e24-fabd-4dfb-81ac-1e93064a230f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:42.265417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23129e24-fabd-4dfb-81ac-1e93064a230f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:42.283057 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-23129e24-fabd-4dfb-81ac-1e93064a230f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:42.284342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-23129e24-fabd-4dfb-81ac-1e93064a230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:42.338244 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-da665567-431a-419b-a454-e7b32514bc0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b60d58f3ec924d219aed4b7ab6dda03a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:42.338850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-da665567-431a-419b-a454-e7b32514bc0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b60d58f3ec924d219aed4b7ab6dda03a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:42.348681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1109/2245] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:43:42.348843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-da665567-431a-419b-a454-e7b32514bc0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b60d58f3ec924d219aed4b7ab6dda03a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:42.355302 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1137/2246] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:43:42 2022] DELETE /identity/v3/users/b60d58f3ec924d219aed4b7ab6dda03a => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:43:42.356556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23129e24-fabd-4dfb-81ac-1e93064a230f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:42.358133 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-23129e24-fabd-4dfb-81ac-1e93064a230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:42.359751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec615143-f782-4140-8eda-b06183989d5b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:42.360047 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec615143-f782-4140-8eda-b06183989d5b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:42.360329 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec615143-f782-4140-8eda-b06183989d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:42.391354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ec615143-f782-4140-8eda-b06183989d5b None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:42.392583 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ec615143-f782-4140-8eda-b06183989d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:42.420933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23129e24-fabd-4dfb-81ac-1e93064a230f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:42.428714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:42.434256 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:42.481903 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30390461-93da-42e6-b965-31a357c11e4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:42.482862 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30390461-93da-42e6-b965-31a357c11e4e service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:42.483211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30390461-93da-42e6-b965-31a357c11e4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:42.483599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30390461-93da-42e6-b965-31a357c11e4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:42.497837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1138/2247] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:42.498204 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30390461-93da-42e6-b965-31a357c11e4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:42.504478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30390461-93da-42e6-b965-31a357c11e4e service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:42.505517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec615143-f782-4140-8eda-b06183989d5b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:42.522951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1110/2248] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:43:42.538016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec615143-f782-4140-8eda-b06183989d5b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:42.543312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a3ba9-359d-4779-83ab-f37f8f8e3d6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:42.544119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a3ba9-359d-4779-83ab-f37f8f8e3d6f service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:42.544625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a3ba9-359d-4779-83ab-f37f8f8e3d6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:42.545008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a3ba9-359d-4779-83ab-f37f8f8e3d6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:42.557502 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a3ba9-359d-4779-83ab-f37f8f8e3d6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:42.559067 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a3ba9-359d-4779-83ab-f37f8f8e3d6f service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:42.573884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1139/2249] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:43:43.090328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30390461-93da-42e6-b965-31a357c11e4e service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:43.091892 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-30390461-93da-42e6-b965-31a357c11e4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:43.108107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30390461-93da-42e6-b965-31a357c11e4e service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:43.114167 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7be85-83b6-4e56-b516-a3e1d852144c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:43.120272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7be85-83b6-4e56-b516-a3e1d852144c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.120272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7be85-83b6-4e56-b516-a3e1d852144c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.120272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7be85-83b6-4e56-b516-a3e1d852144c admin admin] PATH_INFO: `/v3/projects/3fbffa4eae4b4ff89a074930f4e9ab30` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.120272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7be85-83b6-4e56-b516-a3e1d852144c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fbffa4eae4b4ff89a074930f4e9ab30)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:43.122397 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7be85-83b6-4e56-b516-a3e1d852144c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:43.122525 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a3ba9-359d-4779-83ab-f37f8f8e3d6f service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:43.123926 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-492a3ba9-359d-4779-83ab-f37f8f8e3d6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:43.142936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a3ba9-359d-4779-83ab-f37f8f8e3d6f service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:43.155090 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a0f39-ceb7-4c67-a345-75062e8f3fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:43.155893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a0f39-ceb7-4c67-a345-75062e8f3fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.156177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a0f39-ceb7-4c67-a345-75062e8f3fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.156461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a0f39-ceb7-4c67-a345-75062e8f3fc8 admin admin] PATH_INFO: `/v3/projects/ccc77f8e62cd4067bda713897b22e9ff` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.159694 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a0f39-ceb7-4c67-a345-75062e8f3fc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccc77f8e62cd4067bda713897b22e9ff)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:43.161750 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a0f39-ceb7-4c67-a345-75062e8f3fc8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:43.243810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1ce7be85-83b6-4e56-b516-a3e1d852144c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fbffa4eae4b4ff89a074930f4e9ab30', 'request_id': 'req-1ce7be85-83b6-4e56-b516-a3e1d852144c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:43.253735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1ce7be85-83b6-4e56-b516-a3e1d852144c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fbffa4eae4b4ff89a074930f4e9ab30', 'request_id': 'req-1ce7be85-83b6-4e56-b516-a3e1d852144c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:43.261925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1ce7be85-83b6-4e56-b516-a3e1d852144c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fbffa4eae4b4ff89a074930f4e9ab30', 'request_id': 'req-1ce7be85-83b6-4e56-b516-a3e1d852144c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:43.267452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0a2a0f39-ceb7-4c67-a345-75062e8f3fc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccc77f8e62cd4067bda713897b22e9ff', 'request_id': 'req-0a2a0f39-ceb7-4c67-a345-75062e8f3fc8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:43.274235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0a2a0f39-ceb7-4c67-a345-75062e8f3fc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccc77f8e62cd4067bda713897b22e9ff', 'request_id': 'req-0a2a0f39-ceb7-4c67-a345-75062e8f3fc8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:43.329915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1ce7be85-83b6-4e56-b516-a3e1d852144c admin admin] The token cache is being invalidate because project 3fbffa4eae4b4ff89a074930f4e9ab30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:43:43.330214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1ce7be85-83b6-4e56-b516-a3e1d852144c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:43.332030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1ce7be85-83b6-4e56-b516-a3e1d852144c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:43.333648 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1111/2250] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:43:43 2022] DELETE /identity/v3/projects/3fbffa4eae4b4ff89a074930f4e9ab30 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:43.338298 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc731a-b053-4b16-8931-d6c783731716 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.338725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc731a-b053-4b16-8931-d6c783731716 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.339041 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc731a-b053-4b16-8931-d6c783731716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.360998 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-affc731a-b053-4b16-8931-d6c783731716 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:43.360998 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-affc731a-b053-4b16-8931-d6c783731716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:43.413477 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0a2a0f39-ceb7-4c67-a345-75062e8f3fc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccc77f8e62cd4067bda713897b22e9ff', 'request_id': 'req-0a2a0f39-ceb7-4c67-a345-75062e8f3fc8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:43.425721 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0a2a0f39-ceb7-4c67-a345-75062e8f3fc8 admin admin] The token cache is being invalidate because project ccc77f8e62cd4067bda713897b22e9ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:43:43.426020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0a2a0f39-ceb7-4c67-a345-75062e8f3fc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:43.426655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0a2a0f39-ceb7-4c67-a345-75062e8f3fc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:43.428284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1140/2251] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:43:43 2022] DELETE /identity/v3/projects/ccc77f8e62cd4067bda713897b22e9ff => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:43.431705 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1112/2252] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:43:43.433276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb358917-d6ad-4de1-996e-47e5330fc27c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.433573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb358917-d6ad-4de1-996e-47e5330fc27c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.433853 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb358917-d6ad-4de1-996e-47e5330fc27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.436343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315b973-d9ca-4e08-b566-eaf44684a234 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.436641 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315b973-d9ca-4e08-b566-eaf44684a234 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.436929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315b973-d9ca-4e08-b566-eaf44684a234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.453261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-cb358917-d6ad-4de1-996e-47e5330fc27c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:43.454526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cb358917-d6ad-4de1-996e-47e5330fc27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:43.458094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2315b973-d9ca-4e08-b566-eaf44684a234 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:43.459335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2315b973-d9ca-4e08-b566-eaf44684a234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:43.463080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1141/2253] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:43:43.469204 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1113/2254] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:43:43.470754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde92f0-7624-46bf-9b86-8af6a2caa674 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.471049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde92f0-7624-46bf-9b86-8af6a2caa674 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.472019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde92f0-7624-46bf-9b86-8af6a2caa674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.473637 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315b973-d9ca-4e08-b566-eaf44684a234 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:43.484422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315b973-d9ca-4e08-b566-eaf44684a234 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:43.490659 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246abf83-0e6e-4d07-8e65-dfd3730fd574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:43.491325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246abf83-0e6e-4d07-8e65-dfd3730fd574 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.491610 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246abf83-0e6e-4d07-8e65-dfd3730fd574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.491893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246abf83-0e6e-4d07-8e65-dfd3730fd574 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.492635 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246abf83-0e6e-4d07-8e65-dfd3730fd574 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:43:43.492781 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3dde92f0-7624-46bf-9b86-8af6a2caa674 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:43.493044 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246abf83-0e6e-4d07-8e65-dfd3730fd574 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:43.494046 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3dde92f0-7624-46bf-9b86-8af6a2caa674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:43.494707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246abf83-0e6e-4d07-8e65-dfd3730fd574 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:43.502456 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1142/2255] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:43:43.509445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1114/2256] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:43:43.510189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde92f0-7624-46bf-9b86-8af6a2caa674 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:43.513287 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246abf83-0e6e-4d07-8e65-dfd3730fd574 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:43.523849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246abf83-0e6e-4d07-8e65-dfd3730fd574 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:43.524711 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde92f0-7624-46bf-9b86-8af6a2caa674 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:43.528697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946b747-d4f2-4c8f-b049-badeff204030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:43.529394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946b747-d4f2-4c8f-b049-badeff204030 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.529685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946b747-d4f2-4c8f-b049-badeff204030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.529968 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946b747-d4f2-4c8f-b049-badeff204030 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.530657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946b747-d4f2-4c8f-b049-badeff204030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:43.531810 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db1dff-6414-46c3-bc38-6f12a106cdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:43.532503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db1dff-6414-46c3-bc38-6f12a106cdf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.532623 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946b747-d4f2-4c8f-b049-badeff204030 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:43.532800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db1dff-6414-46c3-bc38-6f12a106cdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.533112 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db1dff-6414-46c3-bc38-6f12a106cdf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.534578 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db1dff-6414-46c3-bc38-6f12a106cdf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:43:43.535000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db1dff-6414-46c3-bc38-6f12a106cdf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:43.536673 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db1dff-6414-46c3-bc38-6f12a106cdf4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:43.554795 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1143/2257] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:43:43.555591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db1dff-6414-46c3-bc38-6f12a106cdf4 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:43.566420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db1dff-6414-46c3-bc38-6f12a106cdf4 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:43.571786 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1c895-016e-4153-905e-e6bd607720b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:43.573421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1c895-016e-4153-905e-e6bd607720b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.574241 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1c895-016e-4153-905e-e6bd607720b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.574541 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1c895-016e-4153-905e-e6bd607720b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.575419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1c895-016e-4153-905e-e6bd607720b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:43.577671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1c895-016e-4153-905e-e6bd607720b7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:43.621524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:43.634426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1115/2258] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:43:43 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:43:43.635836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946b747-d4f2-4c8f-b049-badeff204030 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:43.667181 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1144/2259] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:43:43 2022] POST /identity/v3/projects => generated 399 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:43:43.668173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1c895-016e-4153-905e-e6bd607720b7 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:43.739598 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946b747-d4f2-4c8f-b049-badeff204030 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:43.785096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:43.792832 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1c895-016e-4153-905e-e6bd607720b7 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:43.800354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad48856-8cb3-47c5-82f8-47abf4c45035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:43.801749 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad48856-8cb3-47c5-82f8-47abf4c45035 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.802055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad48856-8cb3-47c5-82f8-47abf4c45035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.802335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad48856-8cb3-47c5-82f8-47abf4c45035 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.803043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad48856-8cb3-47c5-82f8-47abf4c45035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:43.805572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad48856-8cb3-47c5-82f8-47abf4c45035 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:43.816466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bb64b-33c7-4b85-b7ec-880051e3acae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:43.817307 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bb64b-33c7-4b85-b7ec-880051e3acae admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.817601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bb64b-33c7-4b85-b7ec-880051e3acae admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.817880 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bb64b-33c7-4b85-b7ec-880051e3acae admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.819153 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6bb64b-33c7-4b85-b7ec-880051e3acae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:43.822048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6bb64b-33c7-4b85-b7ec-880051e3acae admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:43.901573 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1116/2260] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:43:43 2022] POST /identity/v3/users => generated 286 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:43:43.904146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1145/2261] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:43:43 2022] POST /identity/v3/users => generated 300 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:43:43.905587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bb64b-33c7-4b85-b7ec-880051e3acae admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:43.913877 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad48856-8cb3-47c5-82f8-47abf4c45035 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:43.918160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bb64b-33c7-4b85-b7ec-880051e3acae admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:43.923612 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3e789-8e8c-46bb-ac58-62ebb2fb54e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:43.924291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3e789-8e8c-46bb-ac58-62ebb2fb54e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.924596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3e789-8e8c-46bb-ac58-62ebb2fb54e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.925000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3e789-8e8c-46bb-ac58-62ebb2fb54e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.929598 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad48856-8cb3-47c5-82f8-47abf4c45035 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:43.930131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3e789-8e8c-46bb-ac58-62ebb2fb54e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:43.930131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3e789-8e8c-46bb-ac58-62ebb2fb54e7 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:43.930924 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616b7cf-17d2-4c4a-bd39-58dbe7f9d7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:43.931632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616b7cf-17d2-4c4a-bd39-58dbe7f9d7ca admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.931919 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616b7cf-17d2-4c4a-bd39-58dbe7f9d7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.932190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616b7cf-17d2-4c4a-bd39-58dbe7f9d7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.933402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616b7cf-17d2-4c4a-bd39-58dbe7f9d7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:43.934990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616b7cf-17d2-4c4a-bd39-58dbe7f9d7ca admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:43.942234 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1117/2262] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:43:43.945867 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3e789-8e8c-46bb-ac58-62ebb2fb54e7 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:43.950685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1146/2263] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:43:43.955289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616b7cf-17d2-4c4a-bd39-58dbe7f9d7ca admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:43.956845 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3e789-8e8c-46bb-ac58-62ebb2fb54e7 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:43.961666 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec9ebb-77aa-4f87-ab8f-66eb3b755fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:43.962396 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec9ebb-77aa-4f87-ab8f-66eb3b755fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.962556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616b7cf-17d2-4c4a-bd39-58dbe7f9d7ca admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:43.962683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec9ebb-77aa-4f87-ab8f-66eb3b755fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.963057 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec9ebb-77aa-4f87-ab8f-66eb3b755fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.963792 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec9ebb-77aa-4f87-ab8f-66eb3b755fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:43.965655 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec9ebb-77aa-4f87-ab8f-66eb3b755fc6 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:43.968003 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de908a-3cc3-4d56-9f3d-c8c85584dc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:43.968668 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de908a-3cc3-4d56-9f3d-c8c85584dc97 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:43.968955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de908a-3cc3-4d56-9f3d-c8c85584dc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:43.969369 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de908a-3cc3-4d56-9f3d-c8c85584dc97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:43.970093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de908a-3cc3-4d56-9f3d-c8c85584dc97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:43.972601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de908a-3cc3-4d56-9f3d-c8c85584dc97 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:43.980163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1118/2264] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:43:43.983883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1147/2265] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:43:43.984797 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec9ebb-77aa-4f87-ab8f-66eb3b755fc6 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:43.987622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de908a-3cc3-4d56-9f3d-c8c85584dc97 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:43.995813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec9ebb-77aa-4f87-ab8f-66eb3b755fc6 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:43.998026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de908a-3cc3-4d56-9f3d-c8c85584dc97 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:44.002877 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250569b0-d5f5-4efb-81db-1ba163e4cddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:44.003731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb174c41-c17d-4f35-a240-8404bdcba05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:44.004145 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250569b0-d5f5-4efb-81db-1ba163e4cddb admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:44.004488 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb174c41-c17d-4f35-a240-8404bdcba05e admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:44.004706 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250569b0-d5f5-4efb-81db-1ba163e4cddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:44.004789 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250569b0-d5f5-4efb-81db-1ba163e4cddb admin admin] PATH_INFO: `/v3/projects/2816313356e942b7854724684d103a95/users/43a038296c284a6db71b8606b81a753a/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:44.004866 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb174c41-c17d-4f35-a240-8404bdcba05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:44.005042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb174c41-c17d-4f35-a240-8404bdcba05e admin admin] PATH_INFO: `/v3/projects/e918fd89be30442298f7c9029911b06f/users/1d8b1c413e2343158f37cc9821731e0e/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:44.023417 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250569b0-d5f5-4efb-81db-1ba163e4cddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2816313356e942b7854724684d103a95, user_id=43a038296c284a6db71b8606b81a753a, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:44.026665 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250569b0-d5f5-4efb-81db-1ba163e4cddb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:44.036399 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb174c41-c17d-4f35-a240-8404bdcba05e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e918fd89be30442298f7c9029911b06f, user_id=1d8b1c413e2343158f37cc9821731e0e, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:44.039479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb174c41-c17d-4f35-a240-8404bdcba05e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:44.131543 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1119/2266] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:43:43 2022] PUT /identity/v3/projects/e918fd89be30442298f7c9029911b06f/users/1d8b1c413e2343158f37cc9821731e0e/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:44.132420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1148/2267] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:43:43 2022] PUT /identity/v3/projects/2816313356e942b7854724684d103a95/users/43a038296c284a6db71b8606b81a753a/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:43:44.142654 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1cef1-0cfc-4514-a3af-fc01b1f08e4d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:44.142978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1cef1-0cfc-4514-a3af-fc01b1f08e4d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:44.143796 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1cef1-0cfc-4514-a3af-fc01b1f08e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:44.146882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4775e-6b40-41fc-afc0-499f874f6515 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:44.147177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4775e-6b40-41fc-afc0-499f874f6515 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:44.148162 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4775e-6b40-41fc-afc0-499f874f6515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:44.229806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f5f1cef1-0cfc-4514-a3af-fc01b1f08e4d None None] MFA Rules not processed for user `1d8b1c413e2343158f37cc9821731e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:44.231075 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f5f1cef1-0cfc-4514-a3af-fc01b1f08e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:44.233591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fbb4775e-6b40-41fc-afc0-499f874f6515 None None] MFA Rules not processed for user `43a038296c284a6db71b8606b81a753a`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:44.234884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fbb4775e-6b40-41fc-afc0-499f874f6515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:44.285498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:44.317154 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1149/2268] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:44.320623 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1120/2269] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:44.328884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4775e-6b40-41fc-afc0-499f874f6515 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:44.340187 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fbb4775e-6b40-41fc-afc0-499f874f6515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:44.340340 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1cef1-0cfc-4514-a3af-fc01b1f08e4d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:44.340340 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f5f1cef1-0cfc-4514-a3af-fc01b1f08e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:44.397038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1cef1-0cfc-4514-a3af-fc01b1f08e4d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:44.400967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f428754-1dad-4d4b-9d71-a2cdfd7d7b43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:44.401662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f428754-1dad-4d4b-9d71-a2cdfd7d7b43 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:44.401954 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f428754-1dad-4d4b-9d71-a2cdfd7d7b43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:44.402242 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f428754-1dad-4d4b-9d71-a2cdfd7d7b43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:44.403255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4775e-6b40-41fc-afc0-499f874f6515 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:44.408383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c3b88-6023-4e3c-8fa7-89e31efc42fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:44.408383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c3b88-6023-4e3c-8fa7-89e31efc42fb service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:44.408383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c3b88-6023-4e3c-8fa7-89e31efc42fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:44.409184 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c3b88-6023-4e3c-8fa7-89e31efc42fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:44.411546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f428754-1dad-4d4b-9d71-a2cdfd7d7b43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:44.413329 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f428754-1dad-4d4b-9d71-a2cdfd7d7b43 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:44.418579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c3b88-6023-4e3c-8fa7-89e31efc42fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:44.420081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c3b88-6023-4e3c-8fa7-89e31efc42fb service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:44.423184 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1121/2270] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:43:44.431398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1150/2271] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:44.449962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f428754-1dad-4d4b-9d71-a2cdfd7d7b43 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:44.560922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f428754-1dad-4d4b-9d71-a2cdfd7d7b43 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:44.566814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b34d6-7df9-4447-ac34-c6a77f9be2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:44.567707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b34d6-7df9-4447-ac34-c6a77f9be2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:44.567999 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b34d6-7df9-4447-ac34-c6a77f9be2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:44.568286 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b34d6-7df9-4447-ac34-c6a77f9be2ed admin admin] PATH_INFO: `/v3/users/43a038296c284a6db71b8606b81a753a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:44.575019 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b34d6-7df9-4447-ac34-c6a77f9be2ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=43a038296c284a6db71b8606b81a753a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:44.577031 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b34d6-7df9-4447-ac34-c6a77f9be2ed admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:44.779282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-481b34d6-7df9-4447-ac34-c6a77f9be2ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43a038296c284a6db71b8606b81a753a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:44.780279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-481b34d6-7df9-4447-ac34-c6a77f9be2ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43a038296c284a6db71b8606b81a753a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:44.782609 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-481b34d6-7df9-4447-ac34-c6a77f9be2ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43a038296c284a6db71b8606b81a753a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:44.793395 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-481b34d6-7df9-4447-ac34-c6a77f9be2ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43a038296c284a6db71b8606b81a753a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:44.812636 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-481b34d6-7df9-4447-ac34-c6a77f9be2ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43a038296c284a6db71b8606b81a753a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:44.934627 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1122/2272] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:43:44 2022] DELETE /identity/v3/users/43a038296c284a6db71b8606b81a753a => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:44.934974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a2937-47aa-4489-b40f-5bf77a17f18f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:44.934974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a2937-47aa-4489-b40f-5bf77a17f18f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:44.934974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a2937-47aa-4489-b40f-5bf77a17f18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:44.952963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-372a2937-47aa-4489-b40f-5bf77a17f18f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:44.954292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-372a2937-47aa-4489-b40f-5bf77a17f18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:44.984194 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:45.014386 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1151/2273] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:45.024825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b34d6-7df9-4447-ac34-c6a77f9be2ed admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:45.026280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-481b34d6-7df9-4447-ac34-c6a77f9be2ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:45.031627 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:45.085748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b34d6-7df9-4447-ac34-c6a77f9be2ed admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:45.089379 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71374921-97db-4f5e-bb70-a4c502172dce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:45.090042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71374921-97db-4f5e-bb70-a4c502172dce service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:45.090353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71374921-97db-4f5e-bb70-a4c502172dce service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:45.090751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71374921-97db-4f5e-bb70-a4c502172dce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:45.100453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71374921-97db-4f5e-bb70-a4c502172dce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:45.101952 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71374921-97db-4f5e-bb70-a4c502172dce service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:45.112633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1123/2274] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:45.271343 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a2937-47aa-4489-b40f-5bf77a17f18f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:45.274929 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-372a2937-47aa-4489-b40f-5bf77a17f18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:45.299162 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a2937-47aa-4489-b40f-5bf77a17f18f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:45.306282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e395d-83ba-4310-95fb-fe2a7c326cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:45.307372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e395d-83ba-4310-95fb-fe2a7c326cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:45.307758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e395d-83ba-4310-95fb-fe2a7c326cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:45.308961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e395d-83ba-4310-95fb-fe2a7c326cff admin admin] PATH_INFO: `/v3/users/365f708d5f1d44a29b3f8af782931a63` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:45.311677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e395d-83ba-4310-95fb-fe2a7c326cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=365f708d5f1d44a29b3f8af782931a63)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:45.314693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e395d-83ba-4310-95fb-fe2a7c326cff admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:45.635447 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd9e395d-83ba-4310-95fb-fe2a7c326cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '365f708d5f1d44a29b3f8af782931a63'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:45.668373 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd9e395d-83ba-4310-95fb-fe2a7c326cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '365f708d5f1d44a29b3f8af782931a63'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:45.767312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd9e395d-83ba-4310-95fb-fe2a7c326cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '365f708d5f1d44a29b3f8af782931a63'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:45.770640 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd9e395d-83ba-4310-95fb-fe2a7c326cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '365f708d5f1d44a29b3f8af782931a63'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:45.790328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cd9e395d-83ba-4310-95fb-fe2a7c326cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '365f708d5f1d44a29b3f8af782931a63'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:45.810354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1152/2275] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:43:45 2022] DELETE /identity/v3/users/365f708d5f1d44a29b3f8af782931a63 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:45.814555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f829034-bcec-4df4-9ef1-259b4e1bdd64 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:45.815211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f829034-bcec-4df4-9ef1-259b4e1bdd64 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:45.815211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f829034-bcec-4df4-9ef1-259b4e1bdd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:45.840112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2f829034-bcec-4df4-9ef1-259b4e1bdd64 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:45.842495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2f829034-bcec-4df4-9ef1-259b4e1bdd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:45.854704 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e395d-83ba-4310-95fb-fe2a7c326cff admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:45.858356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cd9e395d-83ba-4310-95fb-fe2a7c326cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:45.996813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e395d-83ba-4310-95fb-fe2a7c326cff admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:46.000900 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8976e5b-7a68-4362-a36b-366b5208b32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:46.001968 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8976e5b-7a68-4362-a36b-366b5208b32c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:46.002385 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8976e5b-7a68-4362-a36b-366b5208b32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:46.002768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8976e5b-7a68-4362-a36b-366b5208b32c admin admin] PATH_INFO: `/v3/projects/2816313356e942b7854724684d103a95` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:46.004188 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8976e5b-7a68-4362-a36b-366b5208b32c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2816313356e942b7854724684d103a95)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:46.006725 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8976e5b-7a68-4362-a36b-366b5208b32c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:46.011715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1124/2276] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:43:46.020135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f829034-bcec-4df4-9ef1-259b4e1bdd64 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:46.024936 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2f829034-bcec-4df4-9ef1-259b4e1bdd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:46.141415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:46.143889 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a8976e5b-7a68-4362-a36b-366b5208b32c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2816313356e942b7854724684d103a95', 'request_id': 'req-a8976e5b-7a68-4362-a36b-366b5208b32c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:46.151379 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a8976e5b-7a68-4362-a36b-366b5208b32c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2816313356e942b7854724684d103a95', 'request_id': 'req-a8976e5b-7a68-4362-a36b-366b5208b32c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:46.151745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f829034-bcec-4df4-9ef1-259b4e1bdd64 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:46.236183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a8976e5b-7a68-4362-a36b-366b5208b32c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2816313356e942b7854724684d103a95', 'request_id': 'req-a8976e5b-7a68-4362-a36b-366b5208b32c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:46.250473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afc6c4-016c-4cc4-b356-c017b956600d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:46.251626 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afc6c4-016c-4cc4-b356-c017b956600d service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:46.252179 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a8976e5b-7a68-4362-a36b-366b5208b32c admin admin] The token cache is being invalidate because project 2816313356e942b7854724684d103a95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:43:46.252505 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a8976e5b-7a68-4362-a36b-366b5208b32c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:46.252953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afc6c4-016c-4cc4-b356-c017b956600d service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:46.254253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a8976e5b-7a68-4362-a36b-366b5208b32c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:46.254918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afc6c4-016c-4cc4-b356-c017b956600d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:46.256980 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1153/2277] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:43:45 2022] DELETE /identity/v3/projects/2816313356e942b7854724684d103a95 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:43:46.259030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f4afc6c4-016c-4cc4-b356-c017b956600d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:46.320346 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:46.399372 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4afc6c4-016c-4cc4-b356-c017b956600d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:46.401719 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4afc6c4-016c-4cc4-b356-c017b956600d service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:46.422984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1125/2278] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:47.121276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8976e5b-7a68-4362-a36b-366b5208b32c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:47.121276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a8976e5b-7a68-4362-a36b-366b5208b32c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:47.142835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8976e5b-7a68-4362-a36b-366b5208b32c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:47.148778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e38a7f-0086-47b5-90b4-32668ea14b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:47.149769 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e38a7f-0086-47b5-90b4-32668ea14b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:47.150064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e38a7f-0086-47b5-90b4-32668ea14b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:47.150704 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e38a7f-0086-47b5-90b4-32668ea14b47 admin admin] PATH_INFO: `/v3/projects/1b665c15deb043e69049eb0481603765` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:47.151700 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e38a7f-0086-47b5-90b4-32668ea14b47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b665c15deb043e69049eb0481603765)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:47.153583 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e38a7f-0086-47b5-90b4-32668ea14b47 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:47.302458 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e1e38a7f-0086-47b5-90b4-32668ea14b47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b665c15deb043e69049eb0481603765', 'request_id': 'req-e1e38a7f-0086-47b5-90b4-32668ea14b47'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:47.312257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e1e38a7f-0086-47b5-90b4-32668ea14b47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b665c15deb043e69049eb0481603765', 'request_id': 'req-e1e38a7f-0086-47b5-90b4-32668ea14b47'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:47.407677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e1e38a7f-0086-47b5-90b4-32668ea14b47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b665c15deb043e69049eb0481603765', 'request_id': 'req-e1e38a7f-0086-47b5-90b4-32668ea14b47'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:47.420304 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e1e38a7f-0086-47b5-90b4-32668ea14b47 admin admin] The token cache is being invalidate because project 1b665c15deb043e69049eb0481603765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:43:47.420860 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e1e38a7f-0086-47b5-90b4-32668ea14b47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:47.421530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e1e38a7f-0086-47b5-90b4-32668ea14b47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:47.423363 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1154/2279] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:43:47 2022] DELETE /identity/v3/projects/1b665c15deb043e69049eb0481603765 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:47.427243 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afc6c4-016c-4cc4-b356-c017b956600d service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:47.437717 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afc6c4-016c-4cc4-b356-c017b956600d service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:47.513857 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec09955-2e4f-4aab-977e-f76115834f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:47.514868 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec09955-2e4f-4aab-977e-f76115834f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:47.515257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec09955-2e4f-4aab-977e-f76115834f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:47.515837 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec09955-2e4f-4aab-977e-f76115834f39 admin admin] PATH_INFO: `/v3/users/dfb020c7985443dcb5541126c1552219` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:47.517001 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec09955-2e4f-4aab-977e-f76115834f39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfb020c7985443dcb5541126c1552219)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:47.519966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec09955-2e4f-4aab-977e-f76115834f39 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:47.747980 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0ec09955-2e4f-4aab-977e-f76115834f39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfb020c7985443dcb5541126c1552219'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:47.748703 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0ec09955-2e4f-4aab-977e-f76115834f39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfb020c7985443dcb5541126c1552219'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:47.749422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0ec09955-2e4f-4aab-977e-f76115834f39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfb020c7985443dcb5541126c1552219'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:47.764643 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0ec09955-2e4f-4aab-977e-f76115834f39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfb020c7985443dcb5541126c1552219'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:47.792208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0ec09955-2e4f-4aab-977e-f76115834f39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfb020c7985443dcb5541126c1552219'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:47.886681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1126/2280] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:43:47 2022] DELETE /identity/v3/users/dfb020c7985443dcb5541126c1552219 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:48.466051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e38a7f-0086-47b5-90b4-32668ea14b47 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:48.467486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e1e38a7f-0086-47b5-90b4-32668ea14b47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:48.484434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e38a7f-0086-47b5-90b4-32668ea14b47 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:48.544709 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be7ac9-11ed-40ad-8974-f879b024aa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:48.545863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be7ac9-11ed-40ad-8974-f879b024aa90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:48.546159 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be7ac9-11ed-40ad-8974-f879b024aa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:48.546442 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be7ac9-11ed-40ad-8974-f879b024aa90 admin admin] PATH_INFO: `/v3/projects/7835e375e42b4dd8a1cc131e28baeb36` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:48.547663 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be7ac9-11ed-40ad-8974-f879b024aa90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7835e375e42b4dd8a1cc131e28baeb36)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:48.550857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be7ac9-11ed-40ad-8974-f879b024aa90 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:48.694229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-62be7ac9-11ed-40ad-8974-f879b024aa90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7835e375e42b4dd8a1cc131e28baeb36', 'request_id': 'req-62be7ac9-11ed-40ad-8974-f879b024aa90'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:48.700685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-62be7ac9-11ed-40ad-8974-f879b024aa90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7835e375e42b4dd8a1cc131e28baeb36', 'request_id': 'req-62be7ac9-11ed-40ad-8974-f879b024aa90'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:48.795559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-62be7ac9-11ed-40ad-8974-f879b024aa90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7835e375e42b4dd8a1cc131e28baeb36', 'request_id': 'req-62be7ac9-11ed-40ad-8974-f879b024aa90'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:48.797320 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:48.807849 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-62be7ac9-11ed-40ad-8974-f879b024aa90 admin admin] The token cache is being invalidate because project 7835e375e42b4dd8a1cc131e28baeb36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:43:48.808144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-62be7ac9-11ed-40ad-8974-f879b024aa90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:48.808774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-62be7ac9-11ed-40ad-8974-f879b024aa90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:48.810361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1155/2281] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:43:48 2022] DELETE /identity/v3/projects/7835e375e42b4dd8a1cc131e28baeb36 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:48.815650 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52edd6-9801-4187-83d4-6c3abc24b92b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:48.816119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52edd6-9801-4187-83d4-6c3abc24b92b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:48.816234 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52edd6-9801-4187-83d4-6c3abc24b92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:48.848382 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-8c52edd6-9801-4187-83d4-6c3abc24b92b None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:48.849939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8c52edd6-9801-4187-83d4-6c3abc24b92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:48.880569 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:48.915855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1127/2282] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:48.924016 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dd20d-87f9-4a83-a8a7-82247fffb414 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:48.924415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dd20d-87f9-4a83-a8a7-82247fffb414 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:48.928826 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dd20d-87f9-4a83-a8a7-82247fffb414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:48.955152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8d1dd20d-87f9-4a83-a8a7-82247fffb414 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:48.955152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8d1dd20d-87f9-4a83-a8a7-82247fffb414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:48.955152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1156/2283] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:48.959237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52edd6-9801-4187-83d4-6c3abc24b92b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:48.969397 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52edd6-9801-4187-83d4-6c3abc24b92b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:48.974898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc01edd-49c2-4b41-8c6d-58d81b9c777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:48.975606 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc01edd-49c2-4b41-8c6d-58d81b9c777b admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:48.976032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc01edd-49c2-4b41-8c6d-58d81b9c777b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:48.976180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc01edd-49c2-4b41-8c6d-58d81b9c777b admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:48.976920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc01edd-49c2-4b41-8c6d-58d81b9c777b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:43:48.977289 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc01edd-49c2-4b41-8c6d-58d81b9c777b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:48.978953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc01edd-49c2-4b41-8c6d-58d81b9c777b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:48.991118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1128/2284] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:43:48.995753 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dd20d-87f9-4a83-a8a7-82247fffb414 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:49.006513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dd20d-87f9-4a83-a8a7-82247fffb414 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:49.011250 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0b805-720d-4c87-b3a1-771a8ee3f7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:49.011904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0b805-720d-4c87-b3a1-771a8ee3f7f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:49.012192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0b805-720d-4c87-b3a1-771a8ee3f7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:49.012472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0b805-720d-4c87-b3a1-771a8ee3f7f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:49.013225 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0b805-720d-4c87-b3a1-771a8ee3f7f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:49.015133 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0b805-720d-4c87-b3a1-771a8ee3f7f3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:49.094804 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1157/2285] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:43:48 2022] POST /identity/v3/projects => generated 363 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:43:49.099681 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc01edd-49c2-4b41-8c6d-58d81b9c777b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:49.113859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc01edd-49c2-4b41-8c6d-58d81b9c777b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:49.168553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db0412-922d-4f03-8f17-a5e1d6d02636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:49.169289 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db0412-922d-4f03-8f17-a5e1d6d02636 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:49.169575 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db0412-922d-4f03-8f17-a5e1d6d02636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:49.169851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db0412-922d-4f03-8f17-a5e1d6d02636 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:49.170552 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db0412-922d-4f03-8f17-a5e1d6d02636 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:49.172713 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db0412-922d-4f03-8f17-a5e1d6d02636 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:49.264857 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1129/2286] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:43:49 2022] POST /identity/v3/users => generated 282 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:43:49.269164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0b805-720d-4c87-b3a1-771a8ee3f7f3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:49.279749 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0b805-720d-4c87-b3a1-771a8ee3f7f3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:49.285127 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720ddf5-a7f5-46fb-a9a1-22ef102460ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:49.285883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720ddf5-a7f5-46fb-a9a1-22ef102460ce admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:49.286144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720ddf5-a7f5-46fb-a9a1-22ef102460ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:49.286421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720ddf5-a7f5-46fb-a9a1-22ef102460ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:49.287152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720ddf5-a7f5-46fb-a9a1-22ef102460ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:49.288743 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720ddf5-a7f5-46fb-a9a1-22ef102460ce admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:49.300923 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1158/2287] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:43:49.305224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db0412-922d-4f03-8f17-a5e1d6d02636 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:49.316153 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db0412-922d-4f03-8f17-a5e1d6d02636 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:49.321331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b67982-1c83-4a44-8e0c-699e05139122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:49.322008 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b67982-1c83-4a44-8e0c-699e05139122 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:49.322306 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b67982-1c83-4a44-8e0c-699e05139122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:49.322586 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b67982-1c83-4a44-8e0c-699e05139122 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:49.323332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b67982-1c83-4a44-8e0c-699e05139122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:49.325102 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b67982-1c83-4a44-8e0c-699e05139122 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:49.336189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1130/2288] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:43:49.340579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720ddf5-a7f5-46fb-a9a1-22ef102460ce admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:49.351670 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720ddf5-a7f5-46fb-a9a1-22ef102460ce admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:49.356468 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2eb74-befb-4525-8616-7cfaff8fa30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:49.357446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2eb74-befb-4525-8616-7cfaff8fa30f admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:49.357729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2eb74-befb-4525-8616-7cfaff8fa30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:49.358457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2eb74-befb-4525-8616-7cfaff8fa30f admin admin] PATH_INFO: `/v3/projects/23e6e466a1174447840e8a0e6feb59fe/users/a0d7c1472f2241919c076b17d5603877/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:49.372849 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2eb74-befb-4525-8616-7cfaff8fa30f admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e6e466a1174447840e8a0e6feb59fe, user_id=a0d7c1472f2241919c076b17d5603877, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:49.375576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2eb74-befb-4525-8616-7cfaff8fa30f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:49.494419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1159/2289] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:43:49 2022] PUT /identity/v3/projects/23e6e466a1174447840e8a0e6feb59fe/users/a0d7c1472f2241919c076b17d5603877/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:49.510629 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292fd02-097c-431b-8fd0-2e6b73725819 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:49.510938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292fd02-097c-431b-8fd0-2e6b73725819 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:49.511231 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292fd02-097c-431b-8fd0-2e6b73725819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:49.539563 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a292fd02-097c-431b-8fd0-2e6b73725819 None None] MFA Rules not processed for user `a0d7c1472f2241919c076b17d5603877`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:49.540784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a292fd02-097c-431b-8fd0-2e6b73725819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:49.608566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1131/2290] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:49 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:49.613762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2eb74-befb-4525-8616-7cfaff8fa30f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:49.681823 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2eb74-befb-4525-8616-7cfaff8fa30f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:49.686435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42989a-bc57-479e-98db-04753cf96b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:49.687090 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42989a-bc57-479e-98db-04753cf96b48 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:49.687375 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42989a-bc57-479e-98db-04753cf96b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:49.687650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42989a-bc57-479e-98db-04753cf96b48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:49.688328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42989a-bc57-479e-98db-04753cf96b48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:49.690994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42989a-bc57-479e-98db-04753cf96b48 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:49.768617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1160/2291] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:43:49 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:43:49.772472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292fd02-097c-431b-8fd0-2e6b73725819 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:49.782931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292fd02-097c-431b-8fd0-2e6b73725819 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:49.808042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:49.841150 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33a695-8f42-4c1d-83b5-f1185e759793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:49.841865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33a695-8f42-4c1d-83b5-f1185e759793 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:49.842159 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33a695-8f42-4c1d-83b5-f1185e759793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:49.842445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33a695-8f42-4c1d-83b5-f1185e759793 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:49.843156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33a695-8f42-4c1d-83b5-f1185e759793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:49.845129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33a695-8f42-4c1d-83b5-f1185e759793 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:49.922444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1132/2292] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:43:49 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:43:49.926981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42989a-bc57-479e-98db-04753cf96b48 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:49.937204 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42989a-bc57-479e-98db-04753cf96b48 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:49.942985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c4d98-4df5-461d-a6fc-feea45b976e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:49.943642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c4d98-4df5-461d-a6fc-feea45b976e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:49.943926 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c4d98-4df5-461d-a6fc-feea45b976e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:49.944211 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c4d98-4df5-461d-a6fc-feea45b976e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:49.944932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c4d98-4df5-461d-a6fc-feea45b976e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:49.946546 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c4d98-4df5-461d-a6fc-feea45b976e4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:49.957399 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1161/2293] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:43:49.961628 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33a695-8f42-4c1d-83b5-f1185e759793 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:49.972227 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33a695-8f42-4c1d-83b5-f1185e759793 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:49.976850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182c086-a6fe-4416-8274-f533753f9a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:49.977766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182c086-a6fe-4416-8274-f533753f9a98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:49.978049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182c086-a6fe-4416-8274-f533753f9a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:49.978437 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182c086-a6fe-4416-8274-f533753f9a98 admin admin] PATH_INFO: `/v3/projects/1efa87502b214defbb8efb1e35cb91cd/users/eb9908dadc224786a0b19f7c6a44a146/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:49.993646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182c086-a6fe-4416-8274-f533753f9a98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1efa87502b214defbb8efb1e35cb91cd, user_id=eb9908dadc224786a0b19f7c6a44a146, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:49.996317 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182c086-a6fe-4416-8274-f533753f9a98 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:50.065195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1133/2294] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:43:49 2022] PUT /identity/v3/projects/1efa87502b214defbb8efb1e35cb91cd/users/eb9908dadc224786a0b19f7c6a44a146/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:50.069114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c4d98-4df5-461d-a6fc-feea45b976e4 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:50.082257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c4d98-4df5-461d-a6fc-feea45b976e4 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:50.149423 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d008a-ade6-492f-88d8-09877d7eb3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:50.150198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d008a-ade6-492f-88d8-09877d7eb3b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:50.150486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d008a-ade6-492f-88d8-09877d7eb3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:50.150767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d008a-ade6-492f-88d8-09877d7eb3b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:50.151499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d008a-ade6-492f-88d8-09877d7eb3b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:50.153189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d008a-ade6-492f-88d8-09877d7eb3b5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:50.154873 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:50.171356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1162/2295] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:43:50.173025 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182c086-a6fe-4416-8274-f533753f9a98 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:50.188682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182c086-a6fe-4416-8274-f533753f9a98 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:50.193911 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60254d04-5d0d-4ecd-80eb-af6e6e6d8a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:50.194824 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60254d04-5d0d-4ecd-80eb-af6e6e6d8a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:50.195119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60254d04-5d0d-4ecd-80eb-af6e6e6d8a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:50.195415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60254d04-5d0d-4ecd-80eb-af6e6e6d8a19 admin admin] PATH_INFO: `/v3/projects/1efa87502b214defbb8efb1e35cb91cd/users/eb9908dadc224786a0b19f7c6a44a146/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:50.197640 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60254d04-5d0d-4ecd-80eb-af6e6e6d8a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1efa87502b214defbb8efb1e35cb91cd, user_id=eb9908dadc224786a0b19f7c6a44a146, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:50.200357 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60254d04-5d0d-4ecd-80eb-af6e6e6d8a19 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:50.217744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1134/2296] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:43:50 2022] PUT /identity/v3/projects/1efa87502b214defbb8efb1e35cb91cd/users/eb9908dadc224786a0b19f7c6a44a146/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:50.228649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805ab30-cf45-4b3c-8c37-b9c5f681227a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:50.231905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805ab30-cf45-4b3c-8c37-b9c5f681227a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:50.232238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805ab30-cf45-4b3c-8c37-b9c5f681227a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:50.267567 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2805ab30-cf45-4b3c-8c37-b9c5f681227a None None] MFA Rules not processed for user `eb9908dadc224786a0b19f7c6a44a146`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:50.268931 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2805ab30-cf45-4b3c-8c37-b9c5f681227a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:50.360438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1163/2297] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:50 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:50.370151 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60254d04-5d0d-4ecd-80eb-af6e6e6d8a19 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:50.372267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-60254d04-5d0d-4ecd-80eb-af6e6e6d8a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:50.461133 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60254d04-5d0d-4ecd-80eb-af6e6e6d8a19 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:50.465041 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20c26c-6e85-4c76-aeed-c3319f3eee90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:50.465720 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20c26c-6e85-4c76-aeed-c3319f3eee90 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:50.466018 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20c26c-6e85-4c76-aeed-c3319f3eee90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:50.466286 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20c26c-6e85-4c76-aeed-c3319f3eee90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:50.476926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20c26c-6e85-4c76-aeed-c3319f3eee90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:50.478462 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20c26c-6e85-4c76-aeed-c3319f3eee90 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:50.492622 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1135/2298] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:50 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:50.752166 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805ab30-cf45-4b3c-8c37-b9c5f681227a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:50.764810 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805ab30-cf45-4b3c-8c37-b9c5f681227a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:50.768991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f16108-f739-435b-8ced-4f3dfd56fd55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:50.769722 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f16108-f739-435b-8ced-4f3dfd56fd55 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:50.770007 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f16108-f739-435b-8ced-4f3dfd56fd55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:50.770290 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f16108-f739-435b-8ced-4f3dfd56fd55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:50.844333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f16108-f739-435b-8ced-4f3dfd56fd55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:50.845956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f16108-f739-435b-8ced-4f3dfd56fd55 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:50.859649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1164/2299] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:50 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:53.984111 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20c26c-6e85-4c76-aeed-c3319f3eee90 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:53.986168 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ad20c26c-6e85-4c76-aeed-c3319f3eee90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:54.123750 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20c26c-6e85-4c76-aeed-c3319f3eee90 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:54.130456 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1ca55-9d56-4f68-9fde-3243c1e6473b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:54.131539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1ca55-9d56-4f68-9fde-3243c1e6473b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:54.131935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1ca55-9d56-4f68-9fde-3243c1e6473b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:54.132323 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1ca55-9d56-4f68-9fde-3243c1e6473b admin admin] PATH_INFO: `/v3/users/1d8b1c413e2343158f37cc9821731e0e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:54.134112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1ca55-9d56-4f68-9fde-3243c1e6473b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d8b1c413e2343158f37cc9821731e0e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:54.136223 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1ca55-9d56-4f68-9fde-3243c1e6473b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:54.173204 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:54.358669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fbe1ca55-9d56-4f68-9fde-3243c1e6473b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d8b1c413e2343158f37cc9821731e0e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:54.359330 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fbe1ca55-9d56-4f68-9fde-3243c1e6473b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d8b1c413e2343158f37cc9821731e0e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:54.362180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fbe1ca55-9d56-4f68-9fde-3243c1e6473b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d8b1c413e2343158f37cc9821731e0e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:54.367441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fbe1ca55-9d56-4f68-9fde-3243c1e6473b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d8b1c413e2343158f37cc9821731e0e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:54.389905 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fbe1ca55-9d56-4f68-9fde-3243c1e6473b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d8b1c413e2343158f37cc9821731e0e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:54.466013 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1136/2300] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:43:53 2022] DELETE /identity/v3/users/1d8b1c413e2343158f37cc9821731e0e => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:43:54.470517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae7605-5426-4333-8ca7-b03d291e727d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:54.471051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae7605-5426-4333-8ca7-b03d291e727d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:54.471171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae7605-5426-4333-8ca7-b03d291e727d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:54.489485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-91ae7605-5426-4333-8ca7-b03d291e727d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:54.490689 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-91ae7605-5426-4333-8ca7-b03d291e727d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:54.557703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1165/2301] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:43:54.583198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1ca55-9d56-4f68-9fde-3243c1e6473b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:54.584685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fbe1ca55-9d56-4f68-9fde-3243c1e6473b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:54.672678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1ca55-9d56-4f68-9fde-3243c1e6473b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:54.687583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea0f86-43af-435e-a3e8-e6d938bea9a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:54.688328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea0f86-43af-435e-a3e8-e6d938bea9a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:54.688621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea0f86-43af-435e-a3e8-e6d938bea9a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:54.688955 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea0f86-43af-435e-a3e8-e6d938bea9a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:54.700142 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea0f86-43af-435e-a3e8-e6d938bea9a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:54.701695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea0f86-43af-435e-a3e8-e6d938bea9a7 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:54.714397 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1137/2302] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:43:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:43:55.287014 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae7605-5426-4333-8ca7-b03d291e727d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:43:55.289036 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-91ae7605-5426-4333-8ca7-b03d291e727d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:55.314046 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae7605-5426-4333-8ca7-b03d291e727d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:43:55.320800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf461f5e-92cd-4abe-a33b-845388de5b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:43:55.322596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf461f5e-92cd-4abe-a33b-845388de5b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:55.322779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf461f5e-92cd-4abe-a33b-845388de5b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:55.323048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf461f5e-92cd-4abe-a33b-845388de5b63 admin admin] PATH_INFO: `/v3/projects/e918fd89be30442298f7c9029911b06f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:55.324385 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf461f5e-92cd-4abe-a33b-845388de5b63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e918fd89be30442298f7c9029911b06f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:43:55.326522 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf461f5e-92cd-4abe-a33b-845388de5b63 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:43:55.358566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:55.502719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cf461f5e-92cd-4abe-a33b-845388de5b63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e918fd89be30442298f7c9029911b06f', 'request_id': 'req-cf461f5e-92cd-4abe-a33b-845388de5b63'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:55.524811 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cf461f5e-92cd-4abe-a33b-845388de5b63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e918fd89be30442298f7c9029911b06f', 'request_id': 'req-cf461f5e-92cd-4abe-a33b-845388de5b63'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:55.579068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cf461f5e-92cd-4abe-a33b-845388de5b63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e918fd89be30442298f7c9029911b06f', 'request_id': 'req-cf461f5e-92cd-4abe-a33b-845388de5b63'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:55.593626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cf461f5e-92cd-4abe-a33b-845388de5b63 admin admin] The token cache is being invalidate because project e918fd89be30442298f7c9029911b06f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:43:55.593626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cf461f5e-92cd-4abe-a33b-845388de5b63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:55.593626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cf461f5e-92cd-4abe-a33b-845388de5b63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:43:55.593912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1166/2303] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:43:55 2022] DELETE /identity/v3/projects/e918fd89be30442298f7c9029911b06f => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:43:57.889403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebacc2-4c1c-496e-840d-e20eeee02a7c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:43:57.890005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebacc2-4c1c-496e-840d-e20eeee02a7c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:43:57.890005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebacc2-4c1c-496e-840d-e20eeee02a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:43:57.908299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f7ebacc2-4c1c-496e-840d-e20eeee02a7c None None] MFA Rules not processed for user `a2f1d435732b4ff0a60599309c66eeb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:43:57.909544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f7ebacc2-4c1c-496e-840d-e20eeee02a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:43:57.964343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1138/2304] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:43:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:03.074203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf461f5e-92cd-4abe-a33b-845388de5b63 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:03.075719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cf461f5e-92cd-4abe-a33b-845388de5b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:03.148205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf461f5e-92cd-4abe-a33b-845388de5b63 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:03.153394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19de2d-d9d1-4314-98df-2f366a1a809d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:03.154298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19de2d-d9d1-4314-98df-2f366a1a809d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:03.154590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19de2d-d9d1-4314-98df-2f366a1a809d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:03.159985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19de2d-d9d1-4314-98df-2f366a1a809d admin admin] PATH_INFO: `/v3/users/a0d7c1472f2241919c076b17d5603877` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:03.159985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19de2d-d9d1-4314-98df-2f366a1a809d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0d7c1472f2241919c076b17d5603877)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:03.159985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19de2d-d9d1-4314-98df-2f366a1a809d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:03.359768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1e19de2d-d9d1-4314-98df-2f366a1a809d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0d7c1472f2241919c076b17d5603877'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:03.376130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1e19de2d-d9d1-4314-98df-2f366a1a809d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0d7c1472f2241919c076b17d5603877'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:03.533263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1e19de2d-d9d1-4314-98df-2f366a1a809d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0d7c1472f2241919c076b17d5603877'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:03.533980 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1e19de2d-d9d1-4314-98df-2f366a1a809d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0d7c1472f2241919c076b17d5603877'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:03.542267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1e19de2d-d9d1-4314-98df-2f366a1a809d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0d7c1472f2241919c076b17d5603877'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:03.548654 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1167/2305] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:44:03 2022] DELETE /identity/v3/users/a0d7c1472f2241919c076b17d5603877 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:03.553692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b58c1b-a506-45c7-bec5-efd0a7f39c75 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:03.554192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b58c1b-a506-45c7-bec5-efd0a7f39c75 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:03.554444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b58c1b-a506-45c7-bec5-efd0a7f39c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:03.573148 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-10b58c1b-a506-45c7-bec5-efd0a7f39c75 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:03.574433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-10b58c1b-a506-45c7-bec5-efd0a7f39c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:03.619112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:03.633707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1139/2306] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:03.643246 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19de2d-d9d1-4314-98df-2f366a1a809d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:03.644726 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1e19de2d-d9d1-4314-98df-2f366a1a809d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:03.672094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:03.707347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19de2d-d9d1-4314-98df-2f366a1a809d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:03.711412 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef633f6-c2e0-49f5-9b7b-e10a77e756a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:03.712070 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef633f6-c2e0-49f5-9b7b-e10a77e756a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:03.712354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef633f6-c2e0-49f5-9b7b-e10a77e756a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:03.712658 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef633f6-c2e0-49f5-9b7b-e10a77e756a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:03.722179 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef633f6-c2e0-49f5-9b7b-e10a77e756a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:03.724152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef633f6-c2e0-49f5-9b7b-e10a77e756a3 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:03.734898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1168/2307] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:44:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:44:04.204643 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b58c1b-a506-45c7-bec5-efd0a7f39c75 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:04.206154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-10b58c1b-a506-45c7-bec5-efd0a7f39c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:04.228394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b58c1b-a506-45c7-bec5-efd0a7f39c75 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:04.242694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4839bdf-2753-424d-98ff-60150f72537c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:04.243585 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4839bdf-2753-424d-98ff-60150f72537c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:04.243872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4839bdf-2753-424d-98ff-60150f72537c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:04.244298 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4839bdf-2753-424d-98ff-60150f72537c admin admin] PATH_INFO: `/v3/projects/23e6e466a1174447840e8a0e6feb59fe` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:04.245542 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4839bdf-2753-424d-98ff-60150f72537c admin admin] RBAC: Authorizing `identity:delete_project(project_id=23e6e466a1174447840e8a0e6feb59fe)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:04.247477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4839bdf-2753-424d-98ff-60150f72537c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:04.345487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c4839bdf-2753-424d-98ff-60150f72537c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23e6e466a1174447840e8a0e6feb59fe', 'request_id': 'req-c4839bdf-2753-424d-98ff-60150f72537c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:04.351437 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c4839bdf-2753-424d-98ff-60150f72537c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23e6e466a1174447840e8a0e6feb59fe', 'request_id': 'req-c4839bdf-2753-424d-98ff-60150f72537c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:04.357938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c4839bdf-2753-424d-98ff-60150f72537c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23e6e466a1174447840e8a0e6feb59fe', 'request_id': 'req-c4839bdf-2753-424d-98ff-60150f72537c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:04.458625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c4839bdf-2753-424d-98ff-60150f72537c admin admin] The token cache is being invalidate because project 23e6e466a1174447840e8a0e6feb59fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:44:04.462705 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c4839bdf-2753-424d-98ff-60150f72537c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:04.462705 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c4839bdf-2753-424d-98ff-60150f72537c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:04.462705 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1140/2308] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:44:04 2022] DELETE /identity/v3/projects/23e6e466a1174447840e8a0e6feb59fe => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:04.465192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef633f6-c2e0-49f5-9b7b-e10a77e756a3 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:04.475413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef633f6-c2e0-49f5-9b7b-e10a77e756a3 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:04.535893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d8794-4ef6-4a9d-9180-7e74b0db8d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:04.536774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d8794-4ef6-4a9d-9180-7e74b0db8d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:04.537086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d8794-4ef6-4a9d-9180-7e74b0db8d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:04.537361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d8794-4ef6-4a9d-9180-7e74b0db8d88 admin admin] PATH_INFO: `/v3/users/eb9908dadc224786a0b19f7c6a44a146` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:04.538553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d8794-4ef6-4a9d-9180-7e74b0db8d88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb9908dadc224786a0b19f7c6a44a146)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:04.540461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d8794-4ef6-4a9d-9180-7e74b0db8d88 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:04.713252 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-280d8794-4ef6-4a9d-9180-7e74b0db8d88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb9908dadc224786a0b19f7c6a44a146'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:04.726757 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-280d8794-4ef6-4a9d-9180-7e74b0db8d88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb9908dadc224786a0b19f7c6a44a146'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:04.777856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-280d8794-4ef6-4a9d-9180-7e74b0db8d88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb9908dadc224786a0b19f7c6a44a146'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:04.778642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-280d8794-4ef6-4a9d-9180-7e74b0db8d88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb9908dadc224786a0b19f7c6a44a146'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:04.786909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-280d8794-4ef6-4a9d-9180-7e74b0db8d88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb9908dadc224786a0b19f7c6a44a146'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:04.792904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1169/2309] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:44:04 2022] DELETE /identity/v3/users/eb9908dadc224786a0b19f7c6a44a146 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:05.180451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4839bdf-2753-424d-98ff-60150f72537c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:05.182070 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c4839bdf-2753-424d-98ff-60150f72537c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:05.196095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4839bdf-2753-424d-98ff-60150f72537c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:05.255161 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355340a6-ecfe-4098-9a3d-8960c4d78d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:05.255947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355340a6-ecfe-4098-9a3d-8960c4d78d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:05.256236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355340a6-ecfe-4098-9a3d-8960c4d78d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:05.256517 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355340a6-ecfe-4098-9a3d-8960c4d78d8f admin admin] PATH_INFO: `/v3/projects/1efa87502b214defbb8efb1e35cb91cd` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:05.257873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355340a6-ecfe-4098-9a3d-8960c4d78d8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1efa87502b214defbb8efb1e35cb91cd)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:05.259929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355340a6-ecfe-4098-9a3d-8960c4d78d8f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:05.359545 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-355340a6-ecfe-4098-9a3d-8960c4d78d8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1efa87502b214defbb8efb1e35cb91cd', 'request_id': 'req-355340a6-ecfe-4098-9a3d-8960c4d78d8f'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:05.366226 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-355340a6-ecfe-4098-9a3d-8960c4d78d8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1efa87502b214defbb8efb1e35cb91cd', 'request_id': 'req-355340a6-ecfe-4098-9a3d-8960c4d78d8f'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:05.372861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-355340a6-ecfe-4098-9a3d-8960c4d78d8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1efa87502b214defbb8efb1e35cb91cd', 'request_id': 'req-355340a6-ecfe-4098-9a3d-8960c4d78d8f'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:05.458271 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-355340a6-ecfe-4098-9a3d-8960c4d78d8f admin admin] The token cache is being invalidate because project 1efa87502b214defbb8efb1e35cb91cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:44:05.458673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-355340a6-ecfe-4098-9a3d-8960c4d78d8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:05.459314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-355340a6-ecfe-4098-9a3d-8960c4d78d8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:05.460895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1141/2310] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:44:05 2022] DELETE /identity/v3/projects/1efa87502b214defbb8efb1e35cb91cd => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:05.465843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de16ebb-9e2c-46dc-96d5-6643bac8822e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:05.466744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de16ebb-9e2c-46dc-96d5-6643bac8822e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:05.466744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de16ebb-9e2c-46dc-96d5-6643bac8822e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:05.485138 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7de16ebb-9e2c-46dc-96d5-6643bac8822e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:05.486349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7de16ebb-9e2c-46dc-96d5-6643bac8822e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:05.527698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:05.545598 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1170/2311] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:44:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:05.555175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca04f1-ffc6-4bf8-8763-b2f0de697b3f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:05.555501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca04f1-ffc6-4bf8-8763-b2f0de697b3f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:05.555811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca04f1-ffc6-4bf8-8763-b2f0de697b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:05.562197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:05.576521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d3ca04f1-ffc6-4bf8-8763-b2f0de697b3f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:05.578757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d3ca04f1-ffc6-4bf8-8763-b2f0de697b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:05.589816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1142/2312] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:44:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:05.593972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de16ebb-9e2c-46dc-96d5-6643bac8822e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:05.607322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de16ebb-9e2c-46dc-96d5-6643bac8822e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:05.610841 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35586029-447a-464a-92b5-b14e01beb05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:05.611519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35586029-447a-464a-92b5-b14e01beb05b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:05.611963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35586029-447a-464a-92b5-b14e01beb05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:05.612257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35586029-447a-464a-92b5-b14e01beb05b admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:05.613566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35586029-447a-464a-92b5-b14e01beb05b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:44:05.613886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35586029-447a-464a-92b5-b14e01beb05b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:05.615529 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35586029-447a-464a-92b5-b14e01beb05b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:05.629798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1171/2313] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:44:05.634177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca04f1-ffc6-4bf8-8763-b2f0de697b3f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:05.643502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca04f1-ffc6-4bf8-8763-b2f0de697b3f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:05.649288 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a4076-7211-4de2-ab8c-b3b713f3bd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:05.649961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a4076-7211-4de2-ab8c-b3b713f3bd13 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:05.650258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a4076-7211-4de2-ab8c-b3b713f3bd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:05.650859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a4076-7211-4de2-ab8c-b3b713f3bd13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:05.651568 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a4076-7211-4de2-ab8c-b3b713f3bd13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:05.655004 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a4076-7211-4de2-ab8c-b3b713f3bd13 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:05.783235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1143/2314] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:44:05 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:44:05.785452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35586029-447a-464a-92b5-b14e01beb05b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:05.795285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35586029-447a-464a-92b5-b14e01beb05b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:05.853590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b37b94-cfdd-4a25-84ff-f62d92db3c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:05.854571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b37b94-cfdd-4a25-84ff-f62d92db3c6e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:05.854856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b37b94-cfdd-4a25-84ff-f62d92db3c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:05.855134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b37b94-cfdd-4a25-84ff-f62d92db3c6e admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:05.855820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b37b94-cfdd-4a25-84ff-f62d92db3c6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:05.857758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b37b94-cfdd-4a25-84ff-f62d92db3c6e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:05.921952 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1172/2315] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:44:05 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:44:05.926816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a4076-7211-4de2-ab8c-b3b713f3bd13 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:05.938483 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a4076-7211-4de2-ab8c-b3b713f3bd13 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:05.943082 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8affd9c-9c4f-4012-9aa9-2c2dd31c9326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:05.943759 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8affd9c-9c4f-4012-9aa9-2c2dd31c9326 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:05.944043 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8affd9c-9c4f-4012-9aa9-2c2dd31c9326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:05.944320 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8affd9c-9c4f-4012-9aa9-2c2dd31c9326 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:05.945045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8affd9c-9c4f-4012-9aa9-2c2dd31c9326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:05.946657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8affd9c-9c4f-4012-9aa9-2c2dd31c9326 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:05.956943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1144/2316] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:44:05.961547 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b37b94-cfdd-4a25-84ff-f62d92db3c6e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:05.971747 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b37b94-cfdd-4a25-84ff-f62d92db3c6e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:05.977203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072cbc2-6088-43f5-a186-c736503a7e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:05.977871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072cbc2-6088-43f5-a186-c736503a7e73 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:05.978157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072cbc2-6088-43f5-a186-c736503a7e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:05.978494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072cbc2-6088-43f5-a186-c736503a7e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:05.979181 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072cbc2-6088-43f5-a186-c736503a7e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:05.980762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072cbc2-6088-43f5-a186-c736503a7e73 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:05.991970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1173/2317] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:44:05.996207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8affd9c-9c4f-4012-9aa9-2c2dd31c9326 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:06.006245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8affd9c-9c4f-4012-9aa9-2c2dd31c9326 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:06.010768 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a050e90-d3c9-4aa8-8597-e89242409eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:06.011644 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a050e90-d3c9-4aa8-8597-e89242409eae admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:06.011928 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a050e90-d3c9-4aa8-8597-e89242409eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:06.012211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a050e90-d3c9-4aa8-8597-e89242409eae admin admin] PATH_INFO: `/v3/projects/a7441329b9fa4cf6aa15e7e6192605f1/users/b9645ab39a4a4a188d9514809e54929e/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:06.027525 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a050e90-d3c9-4aa8-8597-e89242409eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7441329b9fa4cf6aa15e7e6192605f1, user_id=b9645ab39a4a4a188d9514809e54929e, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:06.030344 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a050e90-d3c9-4aa8-8597-e89242409eae admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:06.105498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1145/2318] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:44:05 2022] PUT /identity/v3/projects/a7441329b9fa4cf6aa15e7e6192605f1/users/b9645ab39a4a4a188d9514809e54929e/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:44:06.116704 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d8d1d-a842-44c3-875f-1dcf5abcded1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:06.117001 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d8d1d-a842-44c3-875f-1dcf5abcded1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:06.117321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d8d1d-a842-44c3-875f-1dcf5abcded1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:06.145651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7c9d8d1d-a842-44c3-875f-1dcf5abcded1 None None] MFA Rules not processed for user `b9645ab39a4a4a188d9514809e54929e`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:06.146932 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7c9d8d1d-a842-44c3-875f-1dcf5abcded1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:06.213188 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1174/2319] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:44:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:06.217830 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a050e90-d3c9-4aa8-8597-e89242409eae admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:06.279288 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a050e90-d3c9-4aa8-8597-e89242409eae admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:06.284216 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf350c-e08b-43b1-a677-c880073dc6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:06.284868 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf350c-e08b-43b1-a677-c880073dc6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:06.285757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf350c-e08b-43b1-a677-c880073dc6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:06.286028 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf350c-e08b-43b1-a677-c880073dc6a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:06.286745 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf350c-e08b-43b1-a677-c880073dc6a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:06.288673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf350c-e08b-43b1-a677-c880073dc6a3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:06.380334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1146/2320] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:44:06 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:44:06.384475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d8d1d-a842-44c3-875f-1dcf5abcded1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:06.396068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d8d1d-a842-44c3-875f-1dcf5abcded1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:06.438354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:06.459001 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da364d-d301-4706-9387-dbbf70c90502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:06.459782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da364d-d301-4706-9387-dbbf70c90502 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:06.460081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da364d-d301-4706-9387-dbbf70c90502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:06.460362 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da364d-d301-4706-9387-dbbf70c90502 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:06.461090 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da364d-d301-4706-9387-dbbf70c90502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:06.463151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da364d-d301-4706-9387-dbbf70c90502 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:06.571266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1175/2321] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:44:06 2022] POST /identity/v3/users => generated 299 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:44:06.572432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf350c-e08b-43b1-a677-c880073dc6a3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:06.583099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf350c-e08b-43b1-a677-c880073dc6a3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:06.587740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2f629-2d5a-45be-b159-33cd00eedc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:06.588926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f629-2d5a-45be-b159-33cd00eedc7c admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:06.588926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f629-2d5a-45be-b159-33cd00eedc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:06.589181 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f629-2d5a-45be-b159-33cd00eedc7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:06.589858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2f629-2d5a-45be-b159-33cd00eedc7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:06.591465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2f629-2d5a-45be-b159-33cd00eedc7c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:06.604311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1147/2322] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:44:06.612231 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da364d-d301-4706-9387-dbbf70c90502 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:06.624669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da364d-d301-4706-9387-dbbf70c90502 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:06.630250 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c9f36-c41c-4d04-921b-6824942de4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:06.631153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c9f36-c41c-4d04-921b-6824942de4fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:06.631448 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c9f36-c41c-4d04-921b-6824942de4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:06.631724 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c9f36-c41c-4d04-921b-6824942de4fd admin admin] PATH_INFO: `/v3/projects/566615f3041b439385ebaa617f14b6ec/users/b3cb7019dd544a198d3c7f1094c93cf0/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:06.649439 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c9f36-c41c-4d04-921b-6824942de4fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=566615f3041b439385ebaa617f14b6ec, user_id=b3cb7019dd544a198d3c7f1094c93cf0, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:06.652247 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c9f36-c41c-4d04-921b-6824942de4fd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:06.721703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1176/2323] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:44:06 2022] PUT /identity/v3/projects/566615f3041b439385ebaa617f14b6ec/users/b3cb7019dd544a198d3c7f1094c93cf0/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:44:06.727181 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2f629-2d5a-45be-b159-33cd00eedc7c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:06.745274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2f629-2d5a-45be-b159-33cd00eedc7c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:06.815528 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d50db6-51d8-4fa6-baed-e174de180c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:06.816346 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d50db6-51d8-4fa6-baed-e174de180c98 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:06.816643 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d50db6-51d8-4fa6-baed-e174de180c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:06.817015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d50db6-51d8-4fa6-baed-e174de180c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:06.817851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d50db6-51d8-4fa6-baed-e174de180c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:06.819569 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d50db6-51d8-4fa6-baed-e174de180c98 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:06.829757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1148/2324] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:44:06.833964 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c9f36-c41c-4d04-921b-6824942de4fd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:06.844244 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c9f36-c41c-4d04-921b-6824942de4fd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:06.849411 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbfc60-ed06-4d0b-90df-dd608f754c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:06.850310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbfc60-ed06-4d0b-90df-dd608f754c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:06.850599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbfc60-ed06-4d0b-90df-dd608f754c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:06.850882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbfc60-ed06-4d0b-90df-dd608f754c30 admin admin] PATH_INFO: `/v3/projects/566615f3041b439385ebaa617f14b6ec/users/b3cb7019dd544a198d3c7f1094c93cf0/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:06.852886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbfc60-ed06-4d0b-90df-dd608f754c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=566615f3041b439385ebaa617f14b6ec, user_id=b3cb7019dd544a198d3c7f1094c93cf0, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:06.855592 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbfc60-ed06-4d0b-90df-dd608f754c30 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:06.868502 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1177/2325] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:44:06 2022] PUT /identity/v3/projects/566615f3041b439385ebaa617f14b6ec/users/b3cb7019dd544a198d3c7f1094c93cf0/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:06.878970 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b25ce4-7fab-4fcb-b0d7-17669ab0ef6d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:06.879269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b25ce4-7fab-4fcb-b0d7-17669ab0ef6d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:06.879561 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b25ce4-7fab-4fcb-b0d7-17669ab0ef6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:06.899290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:06.909408 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-82b25ce4-7fab-4fcb-b0d7-17669ab0ef6d None None] MFA Rules not processed for user `b3cb7019dd544a198d3c7f1094c93cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:06.911363 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-82b25ce4-7fab-4fcb-b0d7-17669ab0ef6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:06.993131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1149/2326] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:44:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:07.004805 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbfc60-ed06-4d0b-90df-dd608f754c30 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:07.006733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-54fbfc60-ed06-4d0b-90df-dd608f754c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:07.068254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbfc60-ed06-4d0b-90df-dd608f754c30 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:07.072156 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6616fb3-0c92-450f-902e-65c10c126fdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:07.072816 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6616fb3-0c92-450f-902e-65c10c126fdb service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:07.073121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6616fb3-0c92-450f-902e-65c10c126fdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:07.073400 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6616fb3-0c92-450f-902e-65c10c126fdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:07.084354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6616fb3-0c92-450f-902e-65c10c126fdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:07.085814 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6616fb3-0c92-450f-902e-65c10c126fdb service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:07.096602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1178/2327] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:44:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:44:07.174273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b25ce4-7fab-4fcb-b0d7-17669ab0ef6d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:07.260416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b25ce4-7fab-4fcb-b0d7-17669ab0ef6d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:07.266110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370a2bd-54a1-41b2-bb43-4b5acd8fa5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:07.267208 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370a2bd-54a1-41b2-bb43-4b5acd8fa5a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:07.267625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370a2bd-54a1-41b2-bb43-4b5acd8fa5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:07.267976 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370a2bd-54a1-41b2-bb43-4b5acd8fa5a2 admin admin] PATH_INFO: `/v3/users/b9645ab39a4a4a188d9514809e54929e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:07.270285 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5370a2bd-54a1-41b2-bb43-4b5acd8fa5a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9645ab39a4a4a188d9514809e54929e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:07.277458 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5370a2bd-54a1-41b2-bb43-4b5acd8fa5a2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:07.501484 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5370a2bd-54a1-41b2-bb43-4b5acd8fa5a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9645ab39a4a4a188d9514809e54929e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:07.502077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5370a2bd-54a1-41b2-bb43-4b5acd8fa5a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9645ab39a4a4a188d9514809e54929e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:07.502707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5370a2bd-54a1-41b2-bb43-4b5acd8fa5a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9645ab39a4a4a188d9514809e54929e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:07.510045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5370a2bd-54a1-41b2-bb43-4b5acd8fa5a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9645ab39a4a4a188d9514809e54929e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:07.522327 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-5370a2bd-54a1-41b2-bb43-4b5acd8fa5a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9645ab39a4a4a188d9514809e54929e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:07.636091 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1150/2328] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:44:07 2022] DELETE /identity/v3/users/b9645ab39a4a4a188d9514809e54929e => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:07.640718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406ef56-f64f-4aab-8b25-d6c072652db0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:07.641202 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406ef56-f64f-4aab-8b25-d6c072652db0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:07.641393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406ef56-f64f-4aab-8b25-d6c072652db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:07.659425 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e406ef56-f64f-4aab-8b25-d6c072652db0 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:07.661671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e406ef56-f64f-4aab-8b25-d6c072652db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:07.724651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1179/2329] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:07.734365 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370a2bd-54a1-41b2-bb43-4b5acd8fa5a2 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:07.735791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5370a2bd-54a1-41b2-bb43-4b5acd8fa5a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:07.750675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:07.799336 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370a2bd-54a1-41b2-bb43-4b5acd8fa5a2 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:07.803847 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba7908-326c-4ddc-aa0d-905f5b173d5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:07.804505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba7908-326c-4ddc-aa0d-905f5b173d5c service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:07.804822 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba7908-326c-4ddc-aa0d-905f5b173d5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:07.805161 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba7908-326c-4ddc-aa0d-905f5b173d5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:07.815313 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba7908-326c-4ddc-aa0d-905f5b173d5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:07.817312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba7908-326c-4ddc-aa0d-905f5b173d5c service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:07.827649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1151/2330] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:44:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:44:08.324324 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406ef56-f64f-4aab-8b25-d6c072652db0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:08.325908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e406ef56-f64f-4aab-8b25-d6c072652db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:08.350616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406ef56-f64f-4aab-8b25-d6c072652db0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:08.360554 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d93d81-1da2-48ec-925f-cedde7f08449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:08.361913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d93d81-1da2-48ec-925f-cedde7f08449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:08.361913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d93d81-1da2-48ec-925f-cedde7f08449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:08.362147 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d93d81-1da2-48ec-925f-cedde7f08449 admin admin] PATH_INFO: `/v3/projects/a7441329b9fa4cf6aa15e7e6192605f1` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:08.366025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d93d81-1da2-48ec-925f-cedde7f08449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7441329b9fa4cf6aa15e7e6192605f1)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:08.367980 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d93d81-1da2-48ec-925f-cedde7f08449 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:08.505648 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:08.524573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a0d93d81-1da2-48ec-925f-cedde7f08449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7441329b9fa4cf6aa15e7e6192605f1', 'request_id': 'req-a0d93d81-1da2-48ec-925f-cedde7f08449'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:08.531009 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a0d93d81-1da2-48ec-925f-cedde7f08449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7441329b9fa4cf6aa15e7e6192605f1', 'request_id': 'req-a0d93d81-1da2-48ec-925f-cedde7f08449'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:08.607072 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a0d93d81-1da2-48ec-925f-cedde7f08449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7441329b9fa4cf6aa15e7e6192605f1', 'request_id': 'req-a0d93d81-1da2-48ec-925f-cedde7f08449'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:08.617850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a0d93d81-1da2-48ec-925f-cedde7f08449 admin admin] The token cache is being invalidate because project a7441329b9fa4cf6aa15e7e6192605f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:44:08.618336 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a0d93d81-1da2-48ec-925f-cedde7f08449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:08.619361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a0d93d81-1da2-48ec-925f-cedde7f08449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:08.620998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1180/2331] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:44:08 2022] DELETE /identity/v3/projects/a7441329b9fa4cf6aa15e7e6192605f1 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:08.624998 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba7908-326c-4ddc-aa0d-905f5b173d5c service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:08.634726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba7908-326c-4ddc-aa0d-905f5b173d5c service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:08.691777 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecb521-58d5-4ccf-98c7-e78e489a8b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:08.692648 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecb521-58d5-4ccf-98c7-e78e489a8b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:08.692939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecb521-58d5-4ccf-98c7-e78e489a8b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:08.693245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecb521-58d5-4ccf-98c7-e78e489a8b6e admin admin] PATH_INFO: `/v3/users/b3cb7019dd544a198d3c7f1094c93cf0` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:08.694439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecb521-58d5-4ccf-98c7-e78e489a8b6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3cb7019dd544a198d3c7f1094c93cf0)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:08.696375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecb521-58d5-4ccf-98c7-e78e489a8b6e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:08.877789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-38ecb521-58d5-4ccf-98c7-e78e489a8b6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3cb7019dd544a198d3c7f1094c93cf0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:08.878444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-38ecb521-58d5-4ccf-98c7-e78e489a8b6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3cb7019dd544a198d3c7f1094c93cf0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:08.879653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-38ecb521-58d5-4ccf-98c7-e78e489a8b6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3cb7019dd544a198d3c7f1094c93cf0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:08.887128 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-38ecb521-58d5-4ccf-98c7-e78e489a8b6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3cb7019dd544a198d3c7f1094c93cf0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:08.900036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-38ecb521-58d5-4ccf-98c7-e78e489a8b6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3cb7019dd544a198d3c7f1094c93cf0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:09.013015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1152/2332] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:44:08 2022] DELETE /identity/v3/users/b3cb7019dd544a198d3c7f1094c93cf0 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:09.505832 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d93d81-1da2-48ec-925f-cedde7f08449 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:09.508836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a0d93d81-1da2-48ec-925f-cedde7f08449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:09.519953 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d93d81-1da2-48ec-925f-cedde7f08449 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:09.605684 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddbcd6-8b8e-4f07-806f-005573c49202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:09.606587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddbcd6-8b8e-4f07-806f-005573c49202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:09.606841 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddbcd6-8b8e-4f07-806f-005573c49202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:09.607116 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddbcd6-8b8e-4f07-806f-005573c49202 admin admin] PATH_INFO: `/v3/projects/566615f3041b439385ebaa617f14b6ec` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:09.608290 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddbcd6-8b8e-4f07-806f-005573c49202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=566615f3041b439385ebaa617f14b6ec)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:09.610936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddbcd6-8b8e-4f07-806f-005573c49202 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:09.725241 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-35ddbcd6-8b8e-4f07-806f-005573c49202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '566615f3041b439385ebaa617f14b6ec', 'request_id': 'req-35ddbcd6-8b8e-4f07-806f-005573c49202'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:09.731810 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-35ddbcd6-8b8e-4f07-806f-005573c49202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '566615f3041b439385ebaa617f14b6ec', 'request_id': 'req-35ddbcd6-8b8e-4f07-806f-005573c49202'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:09.795990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-35ddbcd6-8b8e-4f07-806f-005573c49202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '566615f3041b439385ebaa617f14b6ec', 'request_id': 'req-35ddbcd6-8b8e-4f07-806f-005573c49202'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:09.808015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-35ddbcd6-8b8e-4f07-806f-005573c49202 admin admin] The token cache is being invalidate because project 566615f3041b439385ebaa617f14b6ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:44:09.808351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-35ddbcd6-8b8e-4f07-806f-005573c49202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:09.809233 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-35ddbcd6-8b8e-4f07-806f-005573c49202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:09.810690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1181/2333] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:44:09 2022] DELETE /identity/v3/projects/566615f3041b439385ebaa617f14b6ec => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:09.815432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0c521-dd57-4c6e-afbc-fec28a4fd1e0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:09.815914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0c521-dd57-4c6e-afbc-fec28a4fd1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:09.816624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0c521-dd57-4c6e-afbc-fec28a4fd1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:09.835519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-bec0c521-dd57-4c6e-afbc-fec28a4fd1e0 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:09.837194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bec0c521-dd57-4c6e-afbc-fec28a4fd1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:09.871951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:09.901104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1153/2334] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:09.909896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622c70a-74da-4f0f-8b80-c6a72db7c489 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:09.910293 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622c70a-74da-4f0f-8b80-c6a72db7c489 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:09.910667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622c70a-74da-4f0f-8b80-c6a72db7c489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:09.930902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f622c70a-74da-4f0f-8b80-c6a72db7c489 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:09.932152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f622c70a-74da-4f0f-8b80-c6a72db7c489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:09.940341 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1182/2335] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:44:09.945271 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0c521-dd57-4c6e-afbc-fec28a4fd1e0 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:09.954864 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0c521-dd57-4c6e-afbc-fec28a4fd1e0 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:09.959579 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361792db-c736-44c5-b6c6-f14d6e42e90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:09.960294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361792db-c736-44c5-b6c6-f14d6e42e90d admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:09.960615 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361792db-c736-44c5-b6c6-f14d6e42e90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:09.960956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361792db-c736-44c5-b6c6-f14d6e42e90d admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:09.961833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361792db-c736-44c5-b6c6-f14d6e42e90d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:44:09.962144 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361792db-c736-44c5-b6c6-f14d6e42e90d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:09.963816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361792db-c736-44c5-b6c6-f14d6e42e90d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:09.975447 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1154/2336] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:44:09.983056 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622c70a-74da-4f0f-8b80-c6a72db7c489 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:09.989307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622c70a-74da-4f0f-8b80-c6a72db7c489 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:09.994056 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be881c-242b-4c2a-8f92-0b53108b62da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:09.994825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be881c-242b-4c2a-8f92-0b53108b62da admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:09.995121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be881c-242b-4c2a-8f92-0b53108b62da admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:09.995401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be881c-242b-4c2a-8f92-0b53108b62da admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:09.996106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be881c-242b-4c2a-8f92-0b53108b62da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:09.998044 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be881c-242b-4c2a-8f92-0b53108b62da admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:10.072524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1183/2337] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:44:09 2022] POST /identity/v3/projects => generated 391 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:44:10.076832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361792db-c736-44c5-b6c6-f14d6e42e90d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:10.087700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361792db-c736-44c5-b6c6-f14d6e42e90d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:10.146562 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001002cd-bad4-4c7e-bdc5-d023a7f2d442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:10.147240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001002cd-bad4-4c7e-bdc5-d023a7f2d442 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:10.147527 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001002cd-bad4-4c7e-bdc5-d023a7f2d442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:10.149627 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001002cd-bad4-4c7e-bdc5-d023a7f2d442 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:10.149627 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001002cd-bad4-4c7e-bdc5-d023a7f2d442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:10.150530 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001002cd-bad4-4c7e-bdc5-d023a7f2d442 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:10.231740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1155/2338] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:44:10 2022] POST /identity/v3/users => generated 296 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:44:10.241410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be881c-242b-4c2a-8f92-0b53108b62da admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:10.247056 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be881c-242b-4c2a-8f92-0b53108b62da admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:10.252037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7f346-6920-4211-8a5f-ea6f115830d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:10.253012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7f346-6920-4211-8a5f-ea6f115830d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:10.253338 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7f346-6920-4211-8a5f-ea6f115830d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:10.253626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7f346-6920-4211-8a5f-ea6f115830d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:10.254379 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7f346-6920-4211-8a5f-ea6f115830d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:10.256006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7f346-6920-4211-8a5f-ea6f115830d5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:10.273594 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1184/2339] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:44:10.280160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001002cd-bad4-4c7e-bdc5-d023a7f2d442 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:10.290974 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001002cd-bad4-4c7e-bdc5-d023a7f2d442 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:10.296791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee7df4-c2c2-46a0-82c7-7195bbd49cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:10.297507 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee7df4-c2c2-46a0-82c7-7195bbd49cbc admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:10.298185 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee7df4-c2c2-46a0-82c7-7195bbd49cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:10.298453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee7df4-c2c2-46a0-82c7-7195bbd49cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:10.299663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee7df4-c2c2-46a0-82c7-7195bbd49cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:10.302137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee7df4-c2c2-46a0-82c7-7195bbd49cbc admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:10.312792 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1156/2340] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:44:10.317509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7f346-6920-4211-8a5f-ea6f115830d5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:10.343527 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7f346-6920-4211-8a5f-ea6f115830d5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:10.350500 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed610f9-5eb7-417c-b81c-1878d444be25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:10.351189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed610f9-5eb7-417c-b81c-1878d444be25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:10.351483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed610f9-5eb7-417c-b81c-1878d444be25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:10.351769 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed610f9-5eb7-417c-b81c-1878d444be25 admin admin] PATH_INFO: `/v3/projects/84988f7dd4f447a68f84997649b25b50/users/6ef28b311fa74c84b10fd5f38e97e7a8/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:10.357989 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:10.371675 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed610f9-5eb7-417c-b81c-1878d444be25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84988f7dd4f447a68f84997649b25b50, user_id=6ef28b311fa74c84b10fd5f38e97e7a8, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:10.374414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed610f9-5eb7-417c-b81c-1878d444be25 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:10.465570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1185/2341] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:44:10 2022] PUT /identity/v3/projects/84988f7dd4f447a68f84997649b25b50/users/6ef28b311fa74c84b10fd5f38e97e7a8/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:44:10.476487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f1f88-190d-4e7f-9e9b-a9950f8b9ef5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:10.476910 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f1f88-190d-4e7f-9e9b-a9950f8b9ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:10.477239 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f1f88-190d-4e7f-9e9b-a9950f8b9ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:10.506118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d75f1f88-190d-4e7f-9e9b-a9950f8b9ef5 None None] MFA Rules not processed for user `6ef28b311fa74c84b10fd5f38e97e7a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:10.507464 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d75f1f88-190d-4e7f-9e9b-a9950f8b9ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:10.600455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1157/2342] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:44:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:10.610908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed610f9-5eb7-417c-b81c-1878d444be25 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:10.703609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed610f9-5eb7-417c-b81c-1878d444be25 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:10.705257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e7ed7-e07c-4d7b-b2f8-8389ce9d32cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:10.705257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e7ed7-e07c-4d7b-b2f8-8389ce9d32cd admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:10.705257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e7ed7-e07c-4d7b-b2f8-8389ce9d32cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:10.705257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e7ed7-e07c-4d7b-b2f8-8389ce9d32cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:10.705840 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e7ed7-e07c-4d7b-b2f8-8389ce9d32cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:10.708760 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e7ed7-e07c-4d7b-b2f8-8389ce9d32cd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:10.799843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1186/2343] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:44:10 2022] POST /identity/v3/projects => generated 393 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:44:10.803987 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f1f88-190d-4e7f-9e9b-a9950f8b9ef5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:10.818767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f1f88-190d-4e7f-9e9b-a9950f8b9ef5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:10.851381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:10.881409 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b21bb-15a3-4cc0-b3fa-428e44562f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:10.882429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b21bb-15a3-4cc0-b3fa-428e44562f32 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:10.883240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b21bb-15a3-4cc0-b3fa-428e44562f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:10.885261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b21bb-15a3-4cc0-b3fa-428e44562f32 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:10.885261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b21bb-15a3-4cc0-b3fa-428e44562f32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:10.886924 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b21bb-15a3-4cc0-b3fa-428e44562f32 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:11.012255 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1158/2344] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:44:10 2022] POST /identity/v3/users => generated 303 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:44:11.013633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e7ed7-e07c-4d7b-b2f8-8389ce9d32cd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:11.027042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e7ed7-e07c-4d7b-b2f8-8389ce9d32cd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:11.032456 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c717e-c4f6-48b2-bf66-f5bceac9f126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:11.033165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c717e-c4f6-48b2-bf66-f5bceac9f126 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:11.033510 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c717e-c4f6-48b2-bf66-f5bceac9f126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:11.033789 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c717e-c4f6-48b2-bf66-f5bceac9f126 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:11.034521 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c717e-c4f6-48b2-bf66-f5bceac9f126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:11.037029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c717e-c4f6-48b2-bf66-f5bceac9f126 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:11.053794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1187/2345] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:44:11.057622 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b21bb-15a3-4cc0-b3fa-428e44562f32 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:11.069350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b21bb-15a3-4cc0-b3fa-428e44562f32 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:11.075835 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6f0f1-34e8-4665-9a13-415516fd3452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:11.079482 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6f0f1-34e8-4665-9a13-415516fd3452 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:11.079482 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6f0f1-34e8-4665-9a13-415516fd3452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:11.079482 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6f0f1-34e8-4665-9a13-415516fd3452 admin admin] PATH_INFO: `/v3/projects/6e32a22947fe456ca5d5c2acbdbcf5f3/users/b04e9317dd1d4a0fb3e785e4a79cbf84/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:11.103172 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6f0f1-34e8-4665-9a13-415516fd3452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e32a22947fe456ca5d5c2acbdbcf5f3, user_id=b04e9317dd1d4a0fb3e785e4a79cbf84, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:11.106118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6f0f1-34e8-4665-9a13-415516fd3452 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:11.172700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1159/2346] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:44:11 2022] PUT /identity/v3/projects/6e32a22947fe456ca5d5c2acbdbcf5f3/users/b04e9317dd1d4a0fb3e785e4a79cbf84/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:11.176695 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c717e-c4f6-48b2-bf66-f5bceac9f126 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:11.187745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c717e-c4f6-48b2-bf66-f5bceac9f126 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:11.253262 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f6e01-8d86-4c9a-8300-b67719078b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:11.254561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f6e01-8d86-4c9a-8300-b67719078b52 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:11.254937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f6e01-8d86-4c9a-8300-b67719078b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:11.255291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f6e01-8d86-4c9a-8300-b67719078b52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:11.256162 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f6e01-8d86-4c9a-8300-b67719078b52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:11.258056 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f6e01-8d86-4c9a-8300-b67719078b52 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:11.270323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1188/2347] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:44:11.274625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6f0f1-34e8-4665-9a13-415516fd3452 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:11.286200 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6f0f1-34e8-4665-9a13-415516fd3452 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:11.292730 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7d12e-480f-4300-b852-cfd617515ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:11.293725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7d12e-480f-4300-b852-cfd617515ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:11.294353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7d12e-480f-4300-b852-cfd617515ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:11.294353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7d12e-480f-4300-b852-cfd617515ef0 admin admin] PATH_INFO: `/v3/projects/6e32a22947fe456ca5d5c2acbdbcf5f3/users/b04e9317dd1d4a0fb3e785e4a79cbf84/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:11.303193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7d12e-480f-4300-b852-cfd617515ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e32a22947fe456ca5d5c2acbdbcf5f3, user_id=b04e9317dd1d4a0fb3e785e4a79cbf84, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:11.303193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7d12e-480f-4300-b852-cfd617515ef0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:11.317167 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1160/2348] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:44:11 2022] PUT /identity/v3/projects/6e32a22947fe456ca5d5c2acbdbcf5f3/users/b04e9317dd1d4a0fb3e785e4a79cbf84/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:11.328461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926211e7-e50c-4fea-8fad-4d7fb0b6e05a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:11.328768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926211e7-e50c-4fea-8fad-4d7fb0b6e05a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:11.329050 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926211e7-e50c-4fea-8fad-4d7fb0b6e05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:11.370370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-926211e7-e50c-4fea-8fad-4d7fb0b6e05a None None] MFA Rules not processed for user `b04e9317dd1d4a0fb3e785e4a79cbf84`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:11.371679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-926211e7-e50c-4fea-8fad-4d7fb0b6e05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:11.419119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:11.446686 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1189/2349] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:44:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:11.458660 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7d12e-480f-4300-b852-cfd617515ef0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:11.460569 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-11c7d12e-480f-4300-b852-cfd617515ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:11.525233 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7d12e-480f-4300-b852-cfd617515ef0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:11.526825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc2a8a-0bb1-41fa-aaeb-27afc2e6584b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:11.527480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc2a8a-0bb1-41fa-aaeb-27afc2e6584b service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:11.527766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc2a8a-0bb1-41fa-aaeb-27afc2e6584b service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:11.528048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc2a8a-0bb1-41fa-aaeb-27afc2e6584b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:11.537640 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc2a8a-0bb1-41fa-aaeb-27afc2e6584b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:11.539118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc2a8a-0bb1-41fa-aaeb-27afc2e6584b service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:11.548737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1161/2350] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:44:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:44:11.593390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926211e7-e50c-4fea-8fad-4d7fb0b6e05a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:11.594792 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-926211e7-e50c-4fea-8fad-4d7fb0b6e05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:11.607243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926211e7-e50c-4fea-8fad-4d7fb0b6e05a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:11.610776 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30173c68-4ebe-4c13-9e5d-cba525dfa1c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:11.611409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30173c68-4ebe-4c13-9e5d-cba525dfa1c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:11.611954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30173c68-4ebe-4c13-9e5d-cba525dfa1c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:11.611954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30173c68-4ebe-4c13-9e5d-cba525dfa1c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:11.676779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30173c68-4ebe-4c13-9e5d-cba525dfa1c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:11.678281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30173c68-4ebe-4c13-9e5d-cba525dfa1c9 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:11.688842 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1190/2351] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:44:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:44:14.023529 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc2a8a-0bb1-41fa-aaeb-27afc2e6584b service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:14.024960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-42cc2a8a-0bb1-41fa-aaeb-27afc2e6584b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:14.096531 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc2a8a-0bb1-41fa-aaeb-27afc2e6584b service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:14.100374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb25a3-b7cc-4a99-b083-44ec9c536492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:14.101043 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb25a3-b7cc-4a99-b083-44ec9c536492 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:14.101474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb25a3-b7cc-4a99-b083-44ec9c536492 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:14.103055 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb25a3-b7cc-4a99-b083-44ec9c536492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:14.103521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0eeb25a3-b7cc-4a99-b083-44ec9c536492 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:14.155904 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:14.184270 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb25a3-b7cc-4a99-b083-44ec9c536492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:14.185761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb25a3-b7cc-4a99-b083-44ec9c536492 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:14.198250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1162/2352] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:44:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:44:16.865278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30173c68-4ebe-4c13-9e5d-cba525dfa1c9 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:16.947260 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30173c68-4ebe-4c13-9e5d-cba525dfa1c9 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:16.952100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ca19d-93fb-43bf-912c-33c82e8f2f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:16.953000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ca19d-93fb-43bf-912c-33c82e8f2f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:16.953314 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ca19d-93fb-43bf-912c-33c82e8f2f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:16.953594 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ca19d-93fb-43bf-912c-33c82e8f2f9f admin admin] PATH_INFO: `/v3/users/6ef28b311fa74c84b10fd5f38e97e7a8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:16.954905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ca19d-93fb-43bf-912c-33c82e8f2f9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ef28b311fa74c84b10fd5f38e97e7a8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:16.957564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ca19d-93fb-43bf-912c-33c82e8f2f9f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:17.109954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d92ca19d-93fb-43bf-912c-33c82e8f2f9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ef28b311fa74c84b10fd5f38e97e7a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:17.126291 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d92ca19d-93fb-43bf-912c-33c82e8f2f9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ef28b311fa74c84b10fd5f38e97e7a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:17.224293 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d92ca19d-93fb-43bf-912c-33c82e8f2f9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ef28b311fa74c84b10fd5f38e97e7a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:17.225103 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d92ca19d-93fb-43bf-912c-33c82e8f2f9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ef28b311fa74c84b10fd5f38e97e7a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:17.236229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d92ca19d-93fb-43bf-912c-33c82e8f2f9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ef28b311fa74c84b10fd5f38e97e7a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:17.238654 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:17.256324 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1191/2353] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:44:16 2022] DELETE /identity/v3/users/6ef28b311fa74c84b10fd5f38e97e7a8 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:17.261314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfba0e0-c73e-4151-b306-aa3fc5d2e510 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:17.262258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfba0e0-c73e-4151-b306-aa3fc5d2e510 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:17.262747 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfba0e0-c73e-4151-b306-aa3fc5d2e510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:17.292278 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-cdfba0e0-c73e-4151-b306-aa3fc5d2e510 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:17.298484 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cdfba0e0-c73e-4151-b306-aa3fc5d2e510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:17.383087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1163/2354] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:17.397972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ca19d-93fb-43bf-912c-33c82e8f2f9f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:17.399487 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d92ca19d-93fb-43bf-912c-33c82e8f2f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:17.472295 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ca19d-93fb-43bf-912c-33c82e8f2f9f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:17.478858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563b8df-9593-41d1-a691-0eb7b19aa863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:17.481244 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563b8df-9593-41d1-a691-0eb7b19aa863 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:17.481244 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563b8df-9593-41d1-a691-0eb7b19aa863 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:17.481244 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563b8df-9593-41d1-a691-0eb7b19aa863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:17.494388 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563b8df-9593-41d1-a691-0eb7b19aa863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:17.496550 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563b8df-9593-41d1-a691-0eb7b19aa863 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:17.511286 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1192/2355] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:44:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:44:18.286186 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfba0e0-c73e-4151-b306-aa3fc5d2e510 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:18.287766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cdfba0e0-c73e-4151-b306-aa3fc5d2e510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:18.299617 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfba0e0-c73e-4151-b306-aa3fc5d2e510 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:18.305806 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f02d-4812-4246-8ea0-a32eada08942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:18.306865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f02d-4812-4246-8ea0-a32eada08942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:18.307332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f02d-4812-4246-8ea0-a32eada08942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:18.307758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f02d-4812-4246-8ea0-a32eada08942 admin admin] PATH_INFO: `/v3/projects/84988f7dd4f447a68f84997649b25b50` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:18.309257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4f02d-4812-4246-8ea0-a32eada08942 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84988f7dd4f447a68f84997649b25b50)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:18.311559 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4f02d-4812-4246-8ea0-a32eada08942 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:18.467103 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-bfc4f02d-4812-4246-8ea0-a32eada08942 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84988f7dd4f447a68f84997649b25b50', 'request_id': 'req-bfc4f02d-4812-4246-8ea0-a32eada08942'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:18.473631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-bfc4f02d-4812-4246-8ea0-a32eada08942 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84988f7dd4f447a68f84997649b25b50', 'request_id': 'req-bfc4f02d-4812-4246-8ea0-a32eada08942'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:18.480186 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-bfc4f02d-4812-4246-8ea0-a32eada08942 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84988f7dd4f447a68f84997649b25b50', 'request_id': 'req-bfc4f02d-4812-4246-8ea0-a32eada08942'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:18.567608 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-bfc4f02d-4812-4246-8ea0-a32eada08942 admin admin] The token cache is being invalidate because project 84988f7dd4f447a68f84997649b25b50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:44:18.568378 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-bfc4f02d-4812-4246-8ea0-a32eada08942 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:18.569525 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-bfc4f02d-4812-4246-8ea0-a32eada08942 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:18.571972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1164/2356] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:44:18 2022] DELETE /identity/v3/projects/84988f7dd4f447a68f84997649b25b50 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:18.578147 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563b8df-9593-41d1-a691-0eb7b19aa863 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:18.593594 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563b8df-9593-41d1-a691-0eb7b19aa863 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:18.657670 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc974d2-058f-4ba1-a0e3-a03b3f2dce03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:18.658567 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc974d2-058f-4ba1-a0e3-a03b3f2dce03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:18.658856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc974d2-058f-4ba1-a0e3-a03b3f2dce03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:18.659136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc974d2-058f-4ba1-a0e3-a03b3f2dce03 admin admin] PATH_INFO: `/v3/users/b04e9317dd1d4a0fb3e785e4a79cbf84` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:18.660886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc974d2-058f-4ba1-a0e3-a03b3f2dce03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b04e9317dd1d4a0fb3e785e4a79cbf84)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:18.663035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc974d2-058f-4ba1-a0e3-a03b3f2dce03 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:18.863649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7dc974d2-058f-4ba1-a0e3-a03b3f2dce03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b04e9317dd1d4a0fb3e785e4a79cbf84'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:18.875449 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7dc974d2-058f-4ba1-a0e3-a03b3f2dce03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b04e9317dd1d4a0fb3e785e4a79cbf84'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:18.966812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7dc974d2-058f-4ba1-a0e3-a03b3f2dce03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b04e9317dd1d4a0fb3e785e4a79cbf84'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:18.967426 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7dc974d2-058f-4ba1-a0e3-a03b3f2dce03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b04e9317dd1d4a0fb3e785e4a79cbf84'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:18.976025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7dc974d2-058f-4ba1-a0e3-a03b3f2dce03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b04e9317dd1d4a0fb3e785e4a79cbf84'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:18.982152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1193/2357] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:44:18 2022] DELETE /identity/v3/users/b04e9317dd1d4a0fb3e785e4a79cbf84 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:19.492829 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f02d-4812-4246-8ea0-a32eada08942 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:19.492829 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bfc4f02d-4812-4246-8ea0-a32eada08942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:19.505278 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f02d-4812-4246-8ea0-a32eada08942 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:19.566739 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7e5d2-d43c-414d-b851-a58e818397fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:19.567609 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7e5d2-d43c-414d-b851-a58e818397fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:19.567894 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7e5d2-d43c-414d-b851-a58e818397fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:19.568177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7e5d2-d43c-414d-b851-a58e818397fd admin admin] PATH_INFO: `/v3/projects/6e32a22947fe456ca5d5c2acbdbcf5f3` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:19.569445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7e5d2-d43c-414d-b851-a58e818397fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e32a22947fe456ca5d5c2acbdbcf5f3)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:19.571399 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7e5d2-d43c-414d-b851-a58e818397fd admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:19.573113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:19.710823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-96f7e5d2-d43c-414d-b851-a58e818397fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e32a22947fe456ca5d5c2acbdbcf5f3', 'request_id': 'req-96f7e5d2-d43c-414d-b851-a58e818397fd'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:19.716172 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-96f7e5d2-d43c-414d-b851-a58e818397fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e32a22947fe456ca5d5c2acbdbcf5f3', 'request_id': 'req-96f7e5d2-d43c-414d-b851-a58e818397fd'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:19.723118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-96f7e5d2-d43c-414d-b851-a58e818397fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e32a22947fe456ca5d5c2acbdbcf5f3', 'request_id': 'req-96f7e5d2-d43c-414d-b851-a58e818397fd'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:19.824150 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-96f7e5d2-d43c-414d-b851-a58e818397fd admin admin] The token cache is being invalidate because project 6e32a22947fe456ca5d5c2acbdbcf5f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:44:19.824454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-96f7e5d2-d43c-414d-b851-a58e818397fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:19.825148 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-96f7e5d2-d43c-414d-b851-a58e818397fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:44:19.826715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1165/2358] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:44:19 2022] DELETE /identity/v3/projects/6e32a22947fe456ca5d5c2acbdbcf5f3 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:44:19.832361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630f6ce-cb1b-4355-9e2a-54e0938ed92b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:19.832870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630f6ce-cb1b-4355-9e2a-54e0938ed92b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:19.833076 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630f6ce-cb1b-4355-9e2a-54e0938ed92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:19.853827 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d630f6ce-cb1b-4355-9e2a-54e0938ed92b None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:19.859025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d630f6ce-cb1b-4355-9e2a-54e0938ed92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:19.876750 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:19.924439 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1194/2359] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:19.933567 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e6bf8-c751-4993-a45f-70ac8c6c097d None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:19.933886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e6bf8-c751-4993-a45f-70ac8c6c097d None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:19.934153 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e6bf8-c751-4993-a45f-70ac8c6c097d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:19.954133 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7c2e6bf8-c751-4993-a45f-70ac8c6c097d None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:19.955377 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7c2e6bf8-c751-4993-a45f-70ac8c6c097d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:19.964461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1166/2360] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:19.968868 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630f6ce-cb1b-4355-9e2a-54e0938ed92b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:19.979977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630f6ce-cb1b-4355-9e2a-54e0938ed92b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:19.986524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed874e29-4e82-4032-83d2-e3cdd6299742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:19.987247 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed874e29-4e82-4032-83d2-e3cdd6299742 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:19.987489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed874e29-4e82-4032-83d2-e3cdd6299742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:19.987831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed874e29-4e82-4032-83d2-e3cdd6299742 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:19.991383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed874e29-4e82-4032-83d2-e3cdd6299742 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:44:19.991383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed874e29-4e82-4032-83d2-e3cdd6299742 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:19.991383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed874e29-4e82-4032-83d2-e3cdd6299742 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:20.002629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1195/2361] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:44:20.007603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e6bf8-c751-4993-a45f-70ac8c6c097d None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:20.016850 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e6bf8-c751-4993-a45f-70ac8c6c097d None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:20.022204 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234de95-85f5-43ae-b96a-21a25860cdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:20.022901 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234de95-85f5-43ae-b96a-21a25860cdbe admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:20.023191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234de95-85f5-43ae-b96a-21a25860cdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:20.023477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234de95-85f5-43ae-b96a-21a25860cdbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:20.024185 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234de95-85f5-43ae-b96a-21a25860cdbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:20.026163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234de95-85f5-43ae-b96a-21a25860cdbe admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:20.148021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1167/2362] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:44:20 2022] POST /identity/v3/projects => generated 371 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:44:20.152192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed874e29-4e82-4032-83d2-e3cdd6299742 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:20.162673 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed874e29-4e82-4032-83d2-e3cdd6299742 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:20.223129 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4468c1a-16c3-4449-acca-2abf6968e568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:20.223129 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4468c1a-16c3-4449-acca-2abf6968e568 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:20.223129 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4468c1a-16c3-4449-acca-2abf6968e568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:20.223129 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4468c1a-16c3-4449-acca-2abf6968e568 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:20.223510 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4468c1a-16c3-4449-acca-2abf6968e568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:20.225488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4468c1a-16c3-4449-acca-2abf6968e568 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:20.350804 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1196/2363] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:44:20 2022] POST /identity/v3/users => generated 286 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:44:20.354906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234de95-85f5-43ae-b96a-21a25860cdbe admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:20.365999 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234de95-85f5-43ae-b96a-21a25860cdbe admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:20.370542 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8f0fb-608c-4bf5-8bfa-7a3f8b4f4e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:20.371201 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8f0fb-608c-4bf5-8bfa-7a3f8b4f4e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:20.371491 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8f0fb-608c-4bf5-8bfa-7a3f8b4f4e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:20.371775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8f0fb-608c-4bf5-8bfa-7a3f8b4f4e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:20.372506 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8f0fb-608c-4bf5-8bfa-7a3f8b4f4e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:20.374164 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8f0fb-608c-4bf5-8bfa-7a3f8b4f4e3d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:20.385465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1168/2364] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:44:20.389484 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4468c1a-16c3-4449-acca-2abf6968e568 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:20.399441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4468c1a-16c3-4449-acca-2abf6968e568 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:20.404124 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01c980-52bf-40da-84b5-a00caa3f8f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:20.404778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01c980-52bf-40da-84b5-a00caa3f8f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:20.405089 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01c980-52bf-40da-84b5-a00caa3f8f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:20.405382 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01c980-52bf-40da-84b5-a00caa3f8f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:20.406138 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01c980-52bf-40da-84b5-a00caa3f8f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:20.407716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01c980-52bf-40da-84b5-a00caa3f8f35 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:20.417974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1197/2365] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:44:20.422254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8f0fb-608c-4bf5-8bfa-7a3f8b4f4e3d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:20.432823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8f0fb-608c-4bf5-8bfa-7a3f8b4f4e3d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:20.437407 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f03b7-d635-4b73-80c7-075b09019325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:20.438280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f03b7-d635-4b73-80c7-075b09019325 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:20.438570 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f03b7-d635-4b73-80c7-075b09019325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:20.438852 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f03b7-d635-4b73-80c7-075b09019325 admin admin] PATH_INFO: `/v3/projects/89df4327f3a748a1ab2a9e7139f7ba37/users/f63f325575e44c4d813085afd3c5090c/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:20.453879 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f03b7-d635-4b73-80c7-075b09019325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89df4327f3a748a1ab2a9e7139f7ba37, user_id=f63f325575e44c4d813085afd3c5090c, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:20.456631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f03b7-d635-4b73-80c7-075b09019325 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:20.576053 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1169/2366] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:44:20 2022] PUT /identity/v3/projects/89df4327f3a748a1ab2a9e7139f7ba37/users/f63f325575e44c4d813085afd3c5090c/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:44:20.584986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37921d10-d441-4cf8-9ef3-bd0c5a022c9b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:20.585335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37921d10-d441-4cf8-9ef3-bd0c5a022c9b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:20.585622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37921d10-d441-4cf8-9ef3-bd0c5a022c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:20.612315 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-37921d10-d441-4cf8-9ef3-bd0c5a022c9b None None] MFA Rules not processed for user `f63f325575e44c4d813085afd3c5090c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:20.614168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-37921d10-d441-4cf8-9ef3-bd0c5a022c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:20.683079 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1198/2367] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:44:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:20.695255 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f03b7-d635-4b73-80c7-075b09019325 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:20.697142 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a53f03b7-d635-4b73-80c7-075b09019325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:20.779274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f03b7-d635-4b73-80c7-075b09019325 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:20.783018 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b45e2b-88fb-4b49-b19a-7a83e22629bf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee7a6c342548f98e7dc73f4f71a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:20.783671 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b45e2b-88fb-4b49-b19a-7a83e22629bf service glance] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:20.783967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b45e2b-88fb-4b49-b19a-7a83e22629bf service glance] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:20.784248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b45e2b-88fb-4b49-b19a-7a83e22629bf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:20.793704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b45e2b-88fb-4b49-b19a-7a83e22629bf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:20.795154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b45e2b-88fb-4b49-b19a-7a83e22629bf service glance] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:20.805438 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1170/2368] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:44:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:44:21.770150 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0ae32-5d1b-4a65-991e-bdfca307b096 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:21.770729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0ae32-5d1b-4a65-991e-bdfca307b096 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:21.770729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0ae32-5d1b-4a65-991e-bdfca307b096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:21.794203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:21.832227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a8f0ae32-5d1b-4a65-991e-bdfca307b096 None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:21.833782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a8f0ae32-5d1b-4a65-991e-bdfca307b096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:21.906411 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1199/2369] 10.222.0.64 () {58 vars in 1009 bytes} [Mon May 23 15:44:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:21.951906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b45e2b-88fb-4b49-b19a-7a83e22629bf service glance] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:21.953498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b2b45e2b-88fb-4b49-b19a-7a83e22629bf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:21.969701 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:22.050314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b45e2b-88fb-4b49-b19a-7a83e22629bf service glance] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:22.055491 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0e3a1-ccf4-4a74-b282-07310d405839 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:22.056203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0e3a1-ccf4-4a74-b282-07310d405839 service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:22.056493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0e3a1-ccf4-4a74-b282-07310d405839 service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:22.056773 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0e3a1-ccf4-4a74-b282-07310d405839 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:22.065488 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0e3a1-ccf4-4a74-b282-07310d405839 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:22.067120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0e3a1-ccf4-4a74-b282-07310d405839 service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:22.075015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1171/2370] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:44:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:44:31.910378 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=40 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:32.071777 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=37 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:50.527904 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ea239-7ade-4738-b685-0eb50afcbff8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:50.528424 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ea239-7ade-4738-b685-0eb50afcbff8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:50.528533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ea239-7ade-4738-b685-0eb50afcbff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:50.547856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-676ea239-7ade-4738-b685-0eb50afcbff8 None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:44:50.549026 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-676ea239-7ade-4738-b685-0eb50afcbff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:44:50.557081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1200/2371] 10.222.0.64 () {58 vars in 967 bytes} [Mon May 23 15:44:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:44:50.572615 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0e3a1-ccf4-4a74-b282-07310d405839 service swift] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:44:50.582170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0e3a1-ccf4-4a74-b282-07310d405839 service swift] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:44:50.587187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f8453-37ba-4fc9-82fe-320ccfd133dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:44:50.587859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f8453-37ba-4fc9-82fe-320ccfd133dc service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:44:50.588147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f8453-37ba-4fc9-82fe-320ccfd133dc service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:44:50.588426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f8453-37ba-4fc9-82fe-320ccfd133dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:44:50.596470 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f8453-37ba-4fc9-82fe-320ccfd133dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:44:50.598591 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f8453-37ba-4fc9-82fe-320ccfd133dc service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:44:50.606664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1172/2372] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:44:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:45:00.211973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ea239-7ade-4738-b685-0eb50afcbff8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:00.214517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-676ea239-7ade-4738-b685-0eb50afcbff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:00.283422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ea239-7ade-4738-b685-0eb50afcbff8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:00.288585 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc5a34-e403-476d-97f8-fa1776e1fad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:00.289487 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc5a34-e403-476d-97f8-fa1776e1fad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:00.289780 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc5a34-e403-476d-97f8-fa1776e1fad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:00.290063 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc5a34-e403-476d-97f8-fa1776e1fad4 admin admin] PATH_INFO: `/v3/users/6badcf8b92dc43a3bc2342e303dec1da` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:00.291329 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc5a34-e403-476d-97f8-fa1776e1fad4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6badcf8b92dc43a3bc2342e303dec1da)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:00.293289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc5a34-e403-476d-97f8-fa1776e1fad4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:00.540870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cefc5a34-e403-476d-97f8-fa1776e1fad4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6badcf8b92dc43a3bc2342e303dec1da'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:00.552990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cefc5a34-e403-476d-97f8-fa1776e1fad4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6badcf8b92dc43a3bc2342e303dec1da'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:00.603295 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=6 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:00.635935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cefc5a34-e403-476d-97f8-fa1776e1fad4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6badcf8b92dc43a3bc2342e303dec1da'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:00.636656 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cefc5a34-e403-476d-97f8-fa1776e1fad4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6badcf8b92dc43a3bc2342e303dec1da'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:00.644736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cefc5a34-e403-476d-97f8-fa1776e1fad4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6badcf8b92dc43a3bc2342e303dec1da'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:00.651339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1201/2373] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:45:00 2022] DELETE /identity/v3/users/6badcf8b92dc43a3bc2342e303dec1da => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:45:00.890480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f8453-37ba-4fc9-82fe-320ccfd133dc service swift] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:00.902297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f8453-37ba-4fc9-82fe-320ccfd133dc service swift] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:00.960840 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbe487-fb09-481e-897f-c80573c8301d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:00.961741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbe487-fb09-481e-897f-c80573c8301d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:00.962032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbe487-fb09-481e-897f-c80573c8301d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:00.962311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbe487-fb09-481e-897f-c80573c8301d admin admin] PATH_INFO: `/v3/projects/c2ef57ea448a49d3af268a02f6f08257` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:00.963481 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbe487-fb09-481e-897f-c80573c8301d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2ef57ea448a49d3af268a02f6f08257)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:00.965480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbe487-fb09-481e-897f-c80573c8301d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:01.060489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6fdbe487-fb09-481e-897f-c80573c8301d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ef57ea448a49d3af268a02f6f08257', 'request_id': 'req-6fdbe487-fb09-481e-897f-c80573c8301d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:01.066427 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6fdbe487-fb09-481e-897f-c80573c8301d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ef57ea448a49d3af268a02f6f08257', 'request_id': 'req-6fdbe487-fb09-481e-897f-c80573c8301d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:01.073485 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6fdbe487-fb09-481e-897f-c80573c8301d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ef57ea448a49d3af268a02f6f08257', 'request_id': 'req-6fdbe487-fb09-481e-897f-c80573c8301d'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:01.151951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6fdbe487-fb09-481e-897f-c80573c8301d admin admin] The token cache is being invalidate because project c2ef57ea448a49d3af268a02f6f08257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:45:01.152432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6fdbe487-fb09-481e-897f-c80573c8301d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:01.152958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6fdbe487-fb09-481e-897f-c80573c8301d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:01.155063 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1173/2374] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:45:00 2022] DELETE /identity/v3/projects/c2ef57ea448a49d3af268a02f6f08257 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:45:01.160796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91007880-6211-4bb7-bc58-01499e8decf6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:01.161180 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91007880-6211-4bb7-bc58-01499e8decf6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:01.161467 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91007880-6211-4bb7-bc58-01499e8decf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:01.181096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-91007880-6211-4bb7-bc58-01499e8decf6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:45:01.186787 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-91007880-6211-4bb7-bc58-01499e8decf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:01.263501 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1202/2375] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:45:01.264353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900382c0-8d6d-41f9-bf6c-a8d1713c8feb None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:01.264353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900382c0-8d6d-41f9-bf6c-a8d1713c8feb None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:01.264353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900382c0-8d6d-41f9-bf6c-a8d1713c8feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:01.284899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-900382c0-8d6d-41f9-bf6c-a8d1713c8feb None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:45:01.286150 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-900382c0-8d6d-41f9-bf6c-a8d1713c8feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:01.294887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1174/2376] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:45:01.299365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91007880-6211-4bb7-bc58-01499e8decf6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:01.309918 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91007880-6211-4bb7-bc58-01499e8decf6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:01.314622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03a66d-56c2-4b3f-804c-afd610ce2502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:01.315285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03a66d-56c2-4b3f-804c-afd610ce2502 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:01.315575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03a66d-56c2-4b3f-804c-afd610ce2502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:01.315860 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03a66d-56c2-4b3f-804c-afd610ce2502 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:01.316591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03a66d-56c2-4b3f-804c-afd610ce2502 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:45:01.316906 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03a66d-56c2-4b3f-804c-afd610ce2502 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:01.318650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03a66d-56c2-4b3f-804c-afd610ce2502 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:01.327412 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:01.332192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1203/2377] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:45:01.335817 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900382c0-8d6d-41f9-bf6c-a8d1713c8feb None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:01.345163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900382c0-8d6d-41f9-bf6c-a8d1713c8feb None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:01.350502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20462c64-a5ba-4669-a730-2b5145030cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:01.351147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20462c64-a5ba-4669-a730-2b5145030cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:01.351435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20462c64-a5ba-4669-a730-2b5145030cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:01.351848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20462c64-a5ba-4669-a730-2b5145030cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:01.352553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20462c64-a5ba-4669-a730-2b5145030cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:01.354808 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20462c64-a5ba-4669-a730-2b5145030cb6 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:01.437521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1175/2378] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:45:01 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:45:01.441804 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03a66d-56c2-4b3f-804c-afd610ce2502 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:01.452154 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03a66d-56c2-4b3f-804c-afd610ce2502 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:01.514248 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925e336-8dfb-48f3-9ced-ed2cfe1799fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:01.514845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925e336-8dfb-48f3-9ced-ed2cfe1799fb admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:01.519717 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925e336-8dfb-48f3-9ced-ed2cfe1799fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:01.519717 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925e336-8dfb-48f3-9ced-ed2cfe1799fb admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:01.519717 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925e336-8dfb-48f3-9ced-ed2cfe1799fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:01.519717 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925e336-8dfb-48f3-9ced-ed2cfe1799fb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:01.610716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1204/2379] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:45:01 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:45:01.612390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20462c64-a5ba-4669-a730-2b5145030cb6 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:01.624992 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20462c64-a5ba-4669-a730-2b5145030cb6 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:01.630280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fae036-dc81-4f48-8512-1deb7d742ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:01.631578 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fae036-dc81-4f48-8512-1deb7d742ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:01.631873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fae036-dc81-4f48-8512-1deb7d742ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:01.632189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fae036-dc81-4f48-8512-1deb7d742ab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:01.632923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fae036-dc81-4f48-8512-1deb7d742ab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:01.634684 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fae036-dc81-4f48-8512-1deb7d742ab6 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:01.645905 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1176/2380] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:45:01.649817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925e336-8dfb-48f3-9ced-ed2cfe1799fb admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:01.660782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925e336-8dfb-48f3-9ced-ed2cfe1799fb admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:01.665793 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4136ea-f15f-46c0-819b-44ab77bd0b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:01.666548 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4136ea-f15f-46c0-819b-44ab77bd0b5a admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:01.666850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4136ea-f15f-46c0-819b-44ab77bd0b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:01.667150 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4136ea-f15f-46c0-819b-44ab77bd0b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:01.667874 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4136ea-f15f-46c0-819b-44ab77bd0b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:01.669719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4136ea-f15f-46c0-819b-44ab77bd0b5a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:01.681493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1205/2381] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:45:01.685728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fae036-dc81-4f48-8512-1deb7d742ab6 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:01.696459 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fae036-dc81-4f48-8512-1deb7d742ab6 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:01.701632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd3f8d-6ee9-4f38-94c3-ced044c39b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:01.702698 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd3f8d-6ee9-4f38-94c3-ced044c39b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:01.703211 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd3f8d-6ee9-4f38-94c3-ced044c39b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:01.703337 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd3f8d-6ee9-4f38-94c3-ced044c39b8c admin admin] PATH_INFO: `/v3/projects/6263581d20ea47bcae22d736f7934e05/users/ba0229c6732541b4a6dda9040c038686/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:01.719124 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd3f8d-6ee9-4f38-94c3-ced044c39b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6263581d20ea47bcae22d736f7934e05, user_id=ba0229c6732541b4a6dda9040c038686, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:01.721899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd3f8d-6ee9-4f38-94c3-ced044c39b8c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:01.812408 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1177/2382] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:45:01 2022] PUT /identity/v3/projects/6263581d20ea47bcae22d736f7934e05/users/ba0229c6732541b4a6dda9040c038686/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:45:01.819201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f520d-3cf0-4d4a-bffb-c46defdc5b30 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:01.819499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f520d-3cf0-4d4a-bffb-c46defdc5b30 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:01.819771 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f520d-3cf0-4d4a-bffb-c46defdc5b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:01.838117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-927f520d-3cf0-4d4a-bffb-c46defdc5b30 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:45:01.839309 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-927f520d-3cf0-4d4a-bffb-c46defdc5b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:01.910474 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1206/2383] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:45:01.921304 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd3f8d-6ee9-4f38-94c3-ced044c39b8c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:01.923353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5cbd3f8d-6ee9-4f38-94c3-ced044c39b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:01.963962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:01.992462 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd3f8d-6ee9-4f38-94c3-ced044c39b8c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:01.996137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b3329-7657-4c4c-8d4f-17228a0d948c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:01.996787 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b3329-7657-4c4c-8d4f-17228a0d948c service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:01.997088 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b3329-7657-4c4c-8d4f-17228a0d948c service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:01.997387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b3329-7657-4c4c-8d4f-17228a0d948c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:02.007338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b3329-7657-4c4c-8d4f-17228a0d948c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:02.008785 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b3329-7657-4c4c-8d4f-17228a0d948c service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:02.019028 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1178/2384] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:45:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:45:02.621348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70336dd7-d7a7-4132-a2fe-f4e74efcf0b4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:02.621914 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70336dd7-d7a7-4132-a2fe-f4e74efcf0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:02.622105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70336dd7-d7a7-4132-a2fe-f4e74efcf0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:02.641577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-70336dd7-d7a7-4132-a2fe-f4e74efcf0b4 None None] MFA Rules not processed for user `472049224bbf48699b8a9410ec61aa9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:45:02.642883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-70336dd7-d7a7-4132-a2fe-f4e74efcf0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:02.718068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1207/2385] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:45:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:45:04.782267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b3329-7657-4c4c-8d4f-17228a0d948c service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:04.783848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-dd8b3329-7657-4c4c-8d4f-17228a0d948c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:04.875690 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b3329-7657-4c4c-8d4f-17228a0d948c service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:04.881757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c2a78-19bc-49dc-97b5-3a56175b1d0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fbaeb7583a456d81cf79f6b101fb1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:04.883298 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c2a78-19bc-49dc-97b5-3a56175b1d0d service placement] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:04.884067 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c2a78-19bc-49dc-97b5-3a56175b1d0d service placement] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:04.884067 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c2a78-19bc-49dc-97b5-3a56175b1d0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:04.890290 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e22c2a78-19bc-49dc-97b5-3a56175b1d0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:04.897208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70336dd7-d7a7-4132-a2fe-f4e74efcf0b4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:04.898292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-70336dd7-d7a7-4132-a2fe-f4e74efcf0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:04.907416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c2a78-19bc-49dc-97b5-3a56175b1d0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:04.909302 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c2a78-19bc-49dc-97b5-3a56175b1d0d service placement] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:04.915760 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70336dd7-d7a7-4132-a2fe-f4e74efcf0b4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:04.921455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f12a55-3e6b-4bf9-ac0b-e0c998ff0f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:04.922948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12a55-3e6b-4bf9-ac0b-e0c998ff0f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:04.923591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12a55-3e6b-4bf9-ac0b-e0c998ff0f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:04.924299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12a55-3e6b-4bf9-ac0b-e0c998ff0f05 admin admin] PATH_INFO: `/v3/users/f63f325575e44c4d813085afd3c5090c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:04.925223 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1179/2386] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:45:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:45:04.927828 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f12a55-3e6b-4bf9-ac0b-e0c998ff0f05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f63f325575e44c4d813085afd3c5090c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:04.931945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f12a55-3e6b-4bf9-ac0b-e0c998ff0f05 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:04.960066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:05.162365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-01f12a55-3e6b-4bf9-ac0b-e0c998ff0f05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f63f325575e44c4d813085afd3c5090c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:05.179082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-01f12a55-3e6b-4bf9-ac0b-e0c998ff0f05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f63f325575e44c4d813085afd3c5090c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:05.276874 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-01f12a55-3e6b-4bf9-ac0b-e0c998ff0f05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f63f325575e44c4d813085afd3c5090c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:05.284210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-01f12a55-3e6b-4bf9-ac0b-e0c998ff0f05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f63f325575e44c4d813085afd3c5090c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:05.285496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-01f12a55-3e6b-4bf9-ac0b-e0c998ff0f05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f63f325575e44c4d813085afd3c5090c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:05.293317 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1208/2387] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:45:04 2022] DELETE /identity/v3/users/f63f325575e44c4d813085afd3c5090c => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:45:05.298197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf39758-3b96-478e-be5a-25ecb8369587 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:05.298499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf39758-3b96-478e-be5a-25ecb8369587 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:05.298878 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf39758-3b96-478e-be5a-25ecb8369587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:05.316805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-aaf39758-3b96-478e-be5a-25ecb8369587 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:45:05.322040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-aaf39758-3b96-478e-be5a-25ecb8369587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:05.387420 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1180/2388] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:45:05.398098 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f12a55-3e6b-4bf9-ac0b-e0c998ff0f05 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:05.400755 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-01f12a55-3e6b-4bf9-ac0b-e0c998ff0f05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:05.498616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f12a55-3e6b-4bf9-ac0b-e0c998ff0f05 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:05.502806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00238ad-c1f3-45e8-8ad1-d85841a063e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:05.503103 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00238ad-c1f3-45e8-8ad1-d85841a063e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:05.503990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00238ad-c1f3-45e8-8ad1-d85841a063e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:05.503990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00238ad-c1f3-45e8-8ad1-d85841a063e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:05.515397 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00238ad-c1f3-45e8-8ad1-d85841a063e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:05.516410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00238ad-c1f3-45e8-8ad1-d85841a063e4 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:05.536925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1209/2389] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:45:06.520789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf39758-3b96-478e-be5a-25ecb8369587 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:06.523245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-aaf39758-3b96-478e-be5a-25ecb8369587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:06.541849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf39758-3b96-478e-be5a-25ecb8369587 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:06.557255 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:06.558326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:06.558607 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:06.558891 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173 admin admin] PATH_INFO: `/v3/projects/89df4327f3a748a1ab2a9e7139f7ba37` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:06.560544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89df4327f3a748a1ab2a9e7139f7ba37)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:06.562550 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:06.716478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:06.741617 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89df4327f3a748a1ab2a9e7139f7ba37', 'request_id': 'req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:06.756557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89df4327f3a748a1ab2a9e7139f7ba37', 'request_id': 'req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:06.763596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89df4327f3a748a1ab2a9e7139f7ba37', 'request_id': 'req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:06.854083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173 admin admin] The token cache is being invalidate because project 89df4327f3a748a1ab2a9e7139f7ba37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:45:06.854660 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:06.856053 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:45:06.859193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1181/2390] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:45:06 2022] DELETE /identity/v3/projects/89df4327f3a748a1ab2a9e7139f7ba37 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:45:08.890886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b9ac6-6c07-45f6-ba5a-8c8340b44dce None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:08.891407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b9ac6-6c07-45f6-ba5a-8c8340b44dce None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:08.891552 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b9ac6-6c07-45f6-ba5a-8c8340b44dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:08.918476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-121b9ac6-6c07-45f6-ba5a-8c8340b44dce None None] MFA Rules not processed for user `ba0229c6732541b4a6dda9040c038686`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:45:08.919637 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-121b9ac6-6c07-45f6-ba5a-8c8340b44dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:08.986292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1210/2391] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:45:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:45:08.991375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:08.992803 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:09.059486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0075c49-5889-4a8b-ba55-f9b5fa2d1173 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:09.064108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b4b15-68d1-4f93-8b8a-d94f6b43faf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:09.064772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b4b15-68d1-4f93-8b8a-d94f6b43faf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:09.065077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b4b15-68d1-4f93-8b8a-d94f6b43faf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:09.065429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b4b15-68d1-4f93-8b8a-d94f6b43faf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:09.066120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b4b15-68d1-4f93-8b8a-d94f6b43faf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:09.068233 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b4b15-68d1-4f93-8b8a-d94f6b43faf6 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:09.158143 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1182/2392] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:45:08 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:45:09.162162 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b9ac6-6c07-45f6-ba5a-8c8340b44dce None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:09.174335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b9ac6-6c07-45f6-ba5a-8c8340b44dce None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:09.202385 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:09.234603 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5bdd8-58ba-480e-8842-70034db91540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:09.235340 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5bdd8-58ba-480e-8842-70034db91540 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:09.235633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5bdd8-58ba-480e-8842-70034db91540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:09.235912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5bdd8-58ba-480e-8842-70034db91540 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:09.236608 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5bdd8-58ba-480e-8842-70034db91540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:09.238655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5bdd8-58ba-480e-8842-70034db91540 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:09.341117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1211/2393] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:45:09 2022] POST /identity/v3/users => generated 296 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:45:09.345042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b4b15-68d1-4f93-8b8a-d94f6b43faf6 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:09.356828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b4b15-68d1-4f93-8b8a-d94f6b43faf6 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:09.368478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f4f37-997f-413b-aa01-051184f2eb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:09.369294 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f4f37-997f-413b-aa01-051184f2eb14 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:09.369663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f4f37-997f-413b-aa01-051184f2eb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:09.370020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f4f37-997f-413b-aa01-051184f2eb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:09.370835 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1f4f37-997f-413b-aa01-051184f2eb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:09.372541 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1f4f37-997f-413b-aa01-051184f2eb14 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:09.388339 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1183/2394] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:45:09.389677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5bdd8-58ba-480e-8842-70034db91540 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:09.401100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5bdd8-58ba-480e-8842-70034db91540 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:09.406538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26982093-7a17-40e3-8983-d2a81cd9793d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:09.407437 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26982093-7a17-40e3-8983-d2a81cd9793d admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:09.414079 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26982093-7a17-40e3-8983-d2a81cd9793d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:09.414079 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26982093-7a17-40e3-8983-d2a81cd9793d admin admin] PATH_INFO: `/v3/projects/561f63bffda54e37a768740ecda8dd52/users/0a10eef43ad9475e8156c62932380057/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:09.425921 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26982093-7a17-40e3-8983-d2a81cd9793d admin admin] RBAC: Authorizing `identity:create_grant(project_id=561f63bffda54e37a768740ecda8dd52, user_id=0a10eef43ad9475e8156c62932380057, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:09.428888 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26982093-7a17-40e3-8983-d2a81cd9793d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:09.516174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1212/2395] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:45:09 2022] PUT /identity/v3/projects/561f63bffda54e37a768740ecda8dd52/users/0a10eef43ad9475e8156c62932380057/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:45:09.520161 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f4f37-997f-413b-aa01-051184f2eb14 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:09.531087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f4f37-997f-413b-aa01-051184f2eb14 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:09.590128 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43601929-25d1-4712-b04b-7a1a1e3fcf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:09.590712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43601929-25d1-4712-b04b-7a1a1e3fcf9d admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:09.591028 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43601929-25d1-4712-b04b-7a1a1e3fcf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:09.591296 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43601929-25d1-4712-b04b-7a1a1e3fcf9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:09.591999 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43601929-25d1-4712-b04b-7a1a1e3fcf9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:09.593672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43601929-25d1-4712-b04b-7a1a1e3fcf9d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:09.606827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1184/2396] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:45:09.608007 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26982093-7a17-40e3-8983-d2a81cd9793d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:09.618866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26982093-7a17-40e3-8983-d2a81cd9793d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:09.624103 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadca60-c7df-41cd-8ea0-d3385dd8bc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:09.625210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadca60-c7df-41cd-8ea0-d3385dd8bc0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:09.625575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadca60-c7df-41cd-8ea0-d3385dd8bc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:09.625931 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadca60-c7df-41cd-8ea0-d3385dd8bc0a admin admin] PATH_INFO: `/v3/projects/561f63bffda54e37a768740ecda8dd52/users/0a10eef43ad9475e8156c62932380057/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:09.628012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadca60-c7df-41cd-8ea0-d3385dd8bc0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=561f63bffda54e37a768740ecda8dd52, user_id=0a10eef43ad9475e8156c62932380057, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:09.631612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadca60-c7df-41cd-8ea0-d3385dd8bc0a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:09.643508 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1213/2397] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:45:09 2022] PUT /identity/v3/projects/561f63bffda54e37a768740ecda8dd52/users/0a10eef43ad9475e8156c62932380057/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:45:15.062964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef6525-8d02-4fbb-afb7-ff2fb038769c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:15.063526 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef6525-8d02-4fbb-afb7-ff2fb038769c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:15.063739 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef6525-8d02-4fbb-afb7-ff2fb038769c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:15.101486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-75ef6525-8d02-4fbb-afb7-ff2fb038769c None None] MFA Rules not processed for user `0a10eef43ad9475e8156c62932380057`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:45:15.103622 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-75ef6525-8d02-4fbb-afb7-ff2fb038769c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:15.129126 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:15.206508 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1185/2398] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:45:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:45:15.222160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadca60-c7df-41cd-8ea0-d3385dd8bc0a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:15.225382 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ccadca60-c7df-41cd-8ea0-d3385dd8bc0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:15.355896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadca60-c7df-41cd-8ea0-d3385dd8bc0a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:15.362235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222db73-b66c-4e9b-a15d-bb45a93bc46f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:15.362913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222db73-b66c-4e9b-a15d-bb45a93bc46f service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:15.363205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222db73-b66c-4e9b-a15d-bb45a93bc46f service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:15.363488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222db73-b66c-4e9b-a15d-bb45a93bc46f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:15.430350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222db73-b66c-4e9b-a15d-bb45a93bc46f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:15.432374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222db73-b66c-4e9b-a15d-bb45a93bc46f service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:15.436130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:15.442564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1214/2399] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:45:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:45:25.492862 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=19 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:26.024372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:35.866725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5692b-77cb-4045-8779-51b9346e8b94 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:35.867288 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5692b-77cb-4045-8779-51b9346e8b94 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:35.867428 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5692b-77cb-4045-8779-51b9346e8b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:35.885804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f4c5692b-77cb-4045-8779-51b9346e8b94 None None] MFA Rules not processed for user `ba0229c6732541b4a6dda9040c038686`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:45:35.886900 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f4c5692b-77cb-4045-8779-51b9346e8b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:35.893772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1186/2400] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:45:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:45:45.879661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:50.347659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be91ff-38aa-44a9-a2ff-a908395ac2f4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:50.348172 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be91ff-38aa-44a9-a2ff-a908395ac2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:50.348277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be91ff-38aa-44a9-a2ff-a908395ac2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:50.365775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-00be91ff-38aa-44a9-a2ff-a908395ac2f4 None None] MFA Rules not processed for user `a2f1d435732b4ff0a60599309c66eeb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:45:50.368498 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-00be91ff-38aa-44a9-a2ff-a908395ac2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:50.423465 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1215/2401] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:45:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:45:57.139147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5692b-77cb-4045-8779-51b9346e8b94 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:45:57.140559 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f4c5692b-77cb-4045-8779-51b9346e8b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:45:57.197147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5692b-77cb-4045-8779-51b9346e8b94 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:45:57.200644 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146643f-ca18-4d15-9279-0b319f8eeea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:45:57.201321 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146643f-ca18-4d15-9279-0b319f8eeea2 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:45:57.201628 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146643f-ca18-4d15-9279-0b319f8eeea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:45:57.201901 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146643f-ca18-4d15-9279-0b319f8eeea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:45:57.210953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146643f-ca18-4d15-9279-0b319f8eeea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:45:57.213422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146643f-ca18-4d15-9279-0b319f8eeea2 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:45:57.222661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1187/2402] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:45:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:46:00.416634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=25 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:07.218527 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=26 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:25.105574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a09ba-522c-4541-b2ba-e9b52095cf3b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:46:25.106158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a09ba-522c-4541-b2ba-e9b52095cf3b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:46:25.106158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a09ba-522c-4541-b2ba-e9b52095cf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:46:25.124844 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-603a09ba-522c-4541-b2ba-e9b52095cf3b None None] MFA Rules not processed for user `5236a93fc3a84d27bf3d150e85eb2589`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:46:25.126088 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-603a09ba-522c-4541-b2ba-e9b52095cf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:46:25.180898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1216/2403] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:46:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:46:33.814651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146643f-ca18-4d15-9279-0b319f8eeea2 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:46:33.823458 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146643f-ca18-4d15-9279-0b319f8eeea2 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:46:33.827391 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d058fa-a395-42b2-9107-86b360d59b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:46:33.828060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d058fa-a395-42b2-9107-86b360d59b4b service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:46:33.828289 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d058fa-a395-42b2-9107-86b360d59b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:46:33.828754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d058fa-a395-42b2-9107-86b360d59b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:46:33.830575 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c9d058fa-a395-42b2-9107-86b360d59b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:46:33.845407 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d058fa-a395-42b2-9107-86b360d59b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:46:33.847018 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d058fa-a395-42b2-9107-86b360d59b4b service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:46:33.858455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1188/2404] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:46:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:46:34.510194 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaccfe-caa2-427b-a0a8-5a0b49de5cdd None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:46:34.510654 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaccfe-caa2-427b-a0a8-5a0b49de5cdd None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:46:34.510751 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaccfe-caa2-427b-a0a8-5a0b49de5cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:46:34.528252 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-61aaccfe-caa2-427b-a0a8-5a0b49de5cdd None None] MFA Rules not processed for user `472049224bbf48699b8a9410ec61aa9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:46:34.529478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-61aaccfe-caa2-427b-a0a8-5a0b49de5cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:46:34.583304 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1217/2405] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:46:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:46:34.819308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d058fa-a395-42b2-9107-86b360d59b4b service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:46:34.828783 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d058fa-a395-42b2-9107-86b360d59b4b service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:46:34.832382 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7dfc1-ae2f-4f75-abdf-9f8cde5736e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:46:34.833018 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7dfc1-ae2f-4f75-abdf-9f8cde5736e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:46:34.837159 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7dfc1-ae2f-4f75-abdf-9f8cde5736e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:46:34.837159 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7dfc1-ae2f-4f75-abdf-9f8cde5736e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:46:34.837159 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d2b7dfc1-ae2f-4f75-abdf-9f8cde5736e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:46:34.849743 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7dfc1-ae2f-4f75-abdf-9f8cde5736e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:46:34.851241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7dfc1-ae2f-4f75-abdf-9f8cde5736e7 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:46:34.862381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1189/2406] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:46:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:46:44.576646 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=50 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:44.857300 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=12 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:46.442215 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaccfe-caa2-427b-a0a8-5a0b49de5cdd None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:46:46.443791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-61aaccfe-caa2-427b-a0a8-5a0b49de5cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:46:46.459783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaccfe-caa2-427b-a0a8-5a0b49de5cdd None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:46:46.468801 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac4b85-53ba-4e12-a8d5-ad410a139a0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:46:46.469495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac4b85-53ba-4e12-a8d5-ad410a139a0e service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:46:46.469788 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac4b85-53ba-4e12-a8d5-ad410a139a0e service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:46:46.470071 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac4b85-53ba-4e12-a8d5-ad410a139a0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:46:46.474093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4bac4b85-53ba-4e12-a8d5-ad410a139a0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:46:46.487162 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac4b85-53ba-4e12-a8d5-ad410a139a0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:46:46.488754 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac4b85-53ba-4e12-a8d5-ad410a139a0e service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:46:46.498180 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1218/2407] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:46:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:46:56.493812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=6 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:03.999379 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7dfc1-ae2f-4f75-abdf-9f8cde5736e7 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:04.000791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d2b7dfc1-ae2f-4f75-abdf-9f8cde5736e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:04.011180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7dfc1-ae2f-4f75-abdf-9f8cde5736e7 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:04.014851 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d4a53-872c-463e-ade0-81ed0669c370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:04.015483 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d4a53-872c-463e-ade0-81ed0669c370 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:04.015767 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d4a53-872c-463e-ade0-81ed0669c370 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:04.016044 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d4a53-872c-463e-ade0-81ed0669c370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:04.017755 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d16d4a53-872c-463e-ade0-81ed0669c370 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:04.082604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d4a53-872c-463e-ade0-81ed0669c370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:04.084056 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d4a53-872c-463e-ade0-81ed0669c370 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:04.093784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1190/2408] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:47:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:04.520042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac4b85-53ba-4e12-a8d5-ad410a139a0e service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:04.529722 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac4b85-53ba-4e12-a8d5-ad410a139a0e service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:04.533355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd493d5f-5540-498c-a5e8-12c9b46d7c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:04.534058 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd493d5f-5540-498c-a5e8-12c9b46d7c14 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:04.534351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd493d5f-5540-498c-a5e8-12c9b46d7c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:04.534637 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd493d5f-5540-498c-a5e8-12c9b46d7c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:04.536590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fd493d5f-5540-498c-a5e8-12c9b46d7c14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:04.546739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd493d5f-5540-498c-a5e8-12c9b46d7c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:04.548218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd493d5f-5540-498c-a5e8-12c9b46d7c14 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:04.557278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1219/2409] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:47:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:04.822008 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48796a2a-fbcd-4b91-9379-cc484962330a None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:04.822220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48796a2a-fbcd-4b91-9379-cc484962330a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:04.822630 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48796a2a-fbcd-4b91-9379-cc484962330a None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:04.824242 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1191/2410] 10.222.0.64 () {54 vars in 921 bytes} [Mon May 23 15:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:04.831871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227aaa2-a959-4631-b9ba-341faa8f5308 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:04.832164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227aaa2-a959-4631-b9ba-341faa8f5308 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:04.832453 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227aaa2-a959-4631-b9ba-341faa8f5308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:04.871584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d227aaa2-a959-4631-b9ba-341faa8f5308 None None] MFA Rules not processed for user `12ceb8a355a44b9fb29918483379cb38`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:04.872846 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d227aaa2-a959-4631-b9ba-341faa8f5308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:04.883464 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1220/2411] 10.222.0.64 () {58 vars in 1018 bytes} [Mon May 23 15:47:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:04.922464 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48796a2a-fbcd-4b91-9379-cc484962330a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:04.940485 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48796a2a-fbcd-4b91-9379-cc484962330a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:04.948561 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a9c93-122d-4bba-a743-b79bc9f626e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:04.948561 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a9c93-122d-4bba-a743-b79bc9f626e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:04.948561 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a9c93-122d-4bba-a743-b79bc9f626e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:04.948561 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a9c93-122d-4bba-a743-b79bc9f626e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:04.956276 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a9c93-122d-4bba-a743-b79bc9f626e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:04.957760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a9c93-122d-4bba-a743-b79bc9f626e6 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:04.965910 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1192/2412] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:47:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:05.079320 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a1cd6-d196-461e-b772-f48a1d121b26 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:05.079559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a1cd6-d196-461e-b772-f48a1d121b26 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:05.079881 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a1cd6-d196-461e-b772-f48a1d121b26 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:05.081709 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1221/2413] 10.222.0.64 () {54 vars in 921 bytes} [Mon May 23 15:47:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:05.087228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2a578-37d0-453b-b8c1-59c71ed377d2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:05.087697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2a578-37d0-453b-b8c1-59c71ed377d2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:05.087831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2a578-37d0-453b-b8c1-59c71ed377d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:05.126440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-71d2a578-37d0-453b-b8c1-59c71ed377d2 None None] MFA Rules not processed for user `8da96bed73794fee8f719ec904936164`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:05.127662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-71d2a578-37d0-453b-b8c1-59c71ed377d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:05.137435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1193/2414] 10.222.0.64 () {58 vars in 1018 bytes} [Mon May 23 15:47:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:05.148824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a1cd6-d196-461e-b772-f48a1d121b26 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:05.160813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a1cd6-d196-461e-b772-f48a1d121b26 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:05.164733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ca524-9606-470a-950e-5c1dc02e0c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:05.165415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ca524-9606-470a-950e-5c1dc02e0c90 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:05.165700 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ca524-9606-470a-950e-5c1dc02e0c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:05.165982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ca524-9606-470a-950e-5c1dc02e0c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:05.177459 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ca524-9606-470a-950e-5c1dc02e0c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:05.178899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ca524-9606-470a-950e-5c1dc02e0c90 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:05.190918 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1222/2415] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:47:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:05.348893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2a578-37d0-453b-b8c1-59c71ed377d2 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:05.350472 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-71d2a578-37d0-453b-b8c1-59c71ed377d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:05.443754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2a578-37d0-453b-b8c1-59c71ed377d2 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:05.449327 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe2b37-a895-4285-9f27-f16ab8f0a960 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fbaeb7583a456d81cf79f6b101fb1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:05.450002 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe2b37-a895-4285-9f27-f16ab8f0a960 service placement] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:05.450348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe2b37-a895-4285-9f27-f16ab8f0a960 service placement] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:05.450634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe2b37-a895-4285-9f27-f16ab8f0a960 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:05.453150 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8bbe2b37-a895-4285-9f27-f16ab8f0a960 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:05.467989 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe2b37-a895-4285-9f27-f16ab8f0a960 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:05.469703 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe2b37-a895-4285-9f27-f16ab8f0a960 service placement] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:05.480117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1194/2416] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:47:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:07.951769 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ca524-9606-470a-950e-5c1dc02e0c90 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:07.964858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ca524-9606-470a-950e-5c1dc02e0c90 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:07.970159 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e6662-a34f-41b8-ac46-b8bdd13f26b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fbaeb7583a456d81cf79f6b101fb1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:07.970836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e6662-a34f-41b8-ac46-b8bdd13f26b0 service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:07.971125 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e6662-a34f-41b8-ac46-b8bdd13f26b0 service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:07.971406 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e6662-a34f-41b8-ac46-b8bdd13f26b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:07.973188 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8e1e6662-a34f-41b8-ac46-b8bdd13f26b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:07.986095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e6662-a34f-41b8-ac46-b8bdd13f26b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:07.987776 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e6662-a34f-41b8-ac46-b8bdd13f26b0 service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:08.000072 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1223/2417] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:47:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:08.059805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe2b37-a895-4285-9f27-f16ab8f0a960 service placement] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:08.062056 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8bbe2b37-a895-4285-9f27-f16ab8f0a960 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:08.075415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe2b37-a895-4285-9f27-f16ab8f0a960 service placement] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:08.080688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52344106-c0b5-4798-ba5a-5ab23f272276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:08.081595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52344106-c0b5-4798-ba5a-5ab23f272276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:08.081881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52344106-c0b5-4798-ba5a-5ab23f272276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:08.082159 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52344106-c0b5-4798-ba5a-5ab23f272276 admin admin] PATH_INFO: `/v3/users/a2f1d435732b4ff0a60599309c66eeb5` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:08.083337 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52344106-c0b5-4798-ba5a-5ab23f272276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2f1d435732b4ff0a60599309c66eeb5)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:08.085302 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52344106-c0b5-4798-ba5a-5ab23f272276 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:08.234586 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e6662-a34f-41b8-ac46-b8bdd13f26b0 service placement] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:08.243839 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e6662-a34f-41b8-ac46-b8bdd13f26b0 service placement] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:08.247488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d23de-0d43-43b5-a9bd-868c555e3ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:08.248134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d23de-0d43-43b5-a9bd-868c555e3ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:08.248480 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d23de-0d43-43b5-a9bd-868c555e3ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:08.248763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d23de-0d43-43b5-a9bd-868c555e3ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:08.250549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-239d23de-0d43-43b5-a9bd-868c555e3ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:08.260319 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d23de-0d43-43b5-a9bd-868c555e3ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:08.261834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d23de-0d43-43b5-a9bd-868c555e3ba6 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:08.270086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1224/2418] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:47:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:47:08.315468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:08.317401 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52344106-c0b5-4798-ba5a-5ab23f272276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2f1d435732b4ff0a60599309c66eeb5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:47:08.318129 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52344106-c0b5-4798-ba5a-5ab23f272276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2f1d435732b4ff0a60599309c66eeb5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:47:08.318844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52344106-c0b5-4798-ba5a-5ab23f272276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2f1d435732b4ff0a60599309c66eeb5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:47:08.326380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52344106-c0b5-4798-ba5a-5ab23f272276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2f1d435732b4ff0a60599309c66eeb5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:47:08.338953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52344106-c0b5-4798-ba5a-5ab23f272276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2f1d435732b4ff0a60599309c66eeb5'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:47:08.429338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1195/2419] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:47:08 2022] DELETE /identity/v3/users/a2f1d435732b4ff0a60599309c66eeb5 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:47:08.670677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d23de-0d43-43b5-a9bd-868c555e3ba6 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:08.703218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d23de-0d43-43b5-a9bd-868c555e3ba6 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:08.761536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3dd81-bd5f-4f31-894d-31af21f5c76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:08.762619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3dd81-bd5f-4f31-894d-31af21f5c76a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:08.762914 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3dd81-bd5f-4f31-894d-31af21f5c76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:08.763200 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3dd81-bd5f-4f31-894d-31af21f5c76a admin admin] PATH_INFO: `/v3/projects/65483f2a0c914022b38e501691095c51` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:08.765131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3dd81-bd5f-4f31-894d-31af21f5c76a admin admin] RBAC: Authorizing `identity:delete_project(project_id=65483f2a0c914022b38e501691095c51)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:08.767086 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3dd81-bd5f-4f31-894d-31af21f5c76a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:08.881998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cbe3dd81-bd5f-4f31-894d-31af21f5c76a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65483f2a0c914022b38e501691095c51', 'request_id': 'req-cbe3dd81-bd5f-4f31-894d-31af21f5c76a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:47:08.888382 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cbe3dd81-bd5f-4f31-894d-31af21f5c76a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65483f2a0c914022b38e501691095c51', 'request_id': 'req-cbe3dd81-bd5f-4f31-894d-31af21f5c76a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:47:08.948720 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cbe3dd81-bd5f-4f31-894d-31af21f5c76a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65483f2a0c914022b38e501691095c51', 'request_id': 'req-cbe3dd81-bd5f-4f31-894d-31af21f5c76a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:47:08.960462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cbe3dd81-bd5f-4f31-894d-31af21f5c76a admin admin] The token cache is being invalidate because project 65483f2a0c914022b38e501691095c51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:47:08.961151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cbe3dd81-bd5f-4f31-894d-31af21f5c76a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:47:08.961665 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cbe3dd81-bd5f-4f31-894d-31af21f5c76a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:47:08.963243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1225/2420] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:47:08 2022] DELETE /identity/v3/projects/65483f2a0c914022b38e501691095c51 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:47:08.968249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e0f28-caf8-405f-9ef9-b0e6036b505e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:08.968600 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e0f28-caf8-405f-9ef9-b0e6036b505e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:08.968894 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e0f28-caf8-405f-9ef9-b0e6036b505e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:08.987030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-577e0f28-caf8-405f-9ef9-b0e6036b505e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:08.988366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-577e0f28-caf8-405f-9ef9-b0e6036b505e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:09.050692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1196/2421] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:09.060462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf831ae8-c65a-4d01-8ecb-36bdd673b2d1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:09.061344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf831ae8-c65a-4d01-8ecb-36bdd673b2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:09.062209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf831ae8-c65a-4d01-8ecb-36bdd673b2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:09.081372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-bf831ae8-c65a-4d01-8ecb-36bdd673b2d1 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:09.082575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bf831ae8-c65a-4d01-8ecb-36bdd673b2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:09.090397 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1226/2422] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:09.095198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e0f28-caf8-405f-9ef9-b0e6036b505e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:09.104784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e0f28-caf8-405f-9ef9-b0e6036b505e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:09.109791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfbc71-9441-4c9e-be8f-22e500853b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:09.110399 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfbc71-9441-4c9e-be8f-22e500853b23 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:09.110690 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfbc71-9441-4c9e-be8f-22e500853b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:09.110976 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfbc71-9441-4c9e-be8f-22e500853b23 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:09.111703 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfbc71-9441-4c9e-be8f-22e500853b23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:47:09.112014 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfbc71-9441-4c9e-be8f-22e500853b23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:09.113670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfbc71-9441-4c9e-be8f-22e500853b23 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:09.125739 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1197/2423] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:47:09.129885 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf831ae8-c65a-4d01-8ecb-36bdd673b2d1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:09.140350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf831ae8-c65a-4d01-8ecb-36bdd673b2d1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:09.144957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79fe88-f88b-4767-b364-ef8a8b7b91e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:09.145634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79fe88-f88b-4767-b364-ef8a8b7b91e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:09.145923 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79fe88-f88b-4767-b364-ef8a8b7b91e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:09.146278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79fe88-f88b-4767-b364-ef8a8b7b91e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:09.146973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79fe88-f88b-4767-b364-ef8a8b7b91e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:09.148867 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79fe88-f88b-4767-b364-ef8a8b7b91e8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:09.157323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:09.247545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1227/2424] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:47:09 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:47:09.252693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfbc71-9441-4c9e-be8f-22e500853b23 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:09.262531 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfbc71-9441-4c9e-be8f-22e500853b23 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:09.321325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5cbdd-e49a-471f-9675-a1cfb14356ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:09.321994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5cbdd-e49a-471f-9675-a1cfb14356ad admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:09.322278 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5cbdd-e49a-471f-9675-a1cfb14356ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:09.322555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5cbdd-e49a-471f-9675-a1cfb14356ad admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:09.323253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5cbdd-e49a-471f-9675-a1cfb14356ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:09.325187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5cbdd-e49a-471f-9675-a1cfb14356ad admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:09.429601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1198/2425] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:47:09 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:47:09.433451 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79fe88-f88b-4767-b364-ef8a8b7b91e8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:09.444433 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79fe88-f88b-4767-b364-ef8a8b7b91e8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:09.449051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bab4b-0b22-4944-83a8-19af125ac1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:09.449728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bab4b-0b22-4944-83a8-19af125ac1c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:09.450015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bab4b-0b22-4944-83a8-19af125ac1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:09.450296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bab4b-0b22-4944-83a8-19af125ac1c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:09.451014 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bab4b-0b22-4944-83a8-19af125ac1c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:09.452745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bab4b-0b22-4944-83a8-19af125ac1c7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:09.462654 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1228/2426] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:47:09.466483 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5cbdd-e49a-471f-9675-a1cfb14356ad admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:09.476352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5cbdd-e49a-471f-9675-a1cfb14356ad admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:09.480932 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d221c4-cf4c-4537-a91f-7124cd00cab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:09.482461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d221c4-cf4c-4537-a91f-7124cd00cab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:09.482461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d221c4-cf4c-4537-a91f-7124cd00cab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:09.482461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d221c4-cf4c-4537-a91f-7124cd00cab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:09.483760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d221c4-cf4c-4537-a91f-7124cd00cab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:09.485342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d221c4-cf4c-4537-a91f-7124cd00cab5 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:09.495234 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1199/2427] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:47:09.500350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bab4b-0b22-4944-83a8-19af125ac1c7 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:09.510216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bab4b-0b22-4944-83a8-19af125ac1c7 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:09.514975 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba7f4b-e883-4d74-919a-972a5cf2d4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:09.515927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba7f4b-e883-4d74-919a-972a5cf2d4c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:09.516214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba7f4b-e883-4d74-919a-972a5cf2d4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:09.516494 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba7f4b-e883-4d74-919a-972a5cf2d4c9 admin admin] PATH_INFO: `/v3/projects/749e4849b23b4e42be64ef0ab0330a0b/users/7a5fc191f3464ffe83d5fde2419cb8b4/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:09.531360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba7f4b-e883-4d74-919a-972a5cf2d4c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=749e4849b23b4e42be64ef0ab0330a0b, user_id=7a5fc191f3464ffe83d5fde2419cb8b4, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:09.534058 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba7f4b-e883-4d74-919a-972a5cf2d4c9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:09.634600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1229/2428] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:47:09 2022] PUT /identity/v3/projects/749e4849b23b4e42be64ef0ab0330a0b/users/7a5fc191f3464ffe83d5fde2419cb8b4/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:47:09.640590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2aa64-072d-4bc6-9ad4-a8c8ce06329b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:09.640590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2aa64-072d-4bc6-9ad4-a8c8ce06329b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:09.640590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2aa64-072d-4bc6-9ad4-a8c8ce06329b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:09.658873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-beb2aa64-072d-4bc6-9ad4-a8c8ce06329b None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:09.660091 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-beb2aa64-072d-4bc6-9ad4-a8c8ce06329b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:09.685076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:09.722489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1200/2429] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:09.736017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba7f4b-e883-4d74-919a-972a5cf2d4c9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:09.737806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-85ba7f4b-e883-4d74-919a-972a5cf2d4c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:09.797276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba7f4b-e883-4d74-919a-972a5cf2d4c9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:09.801963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaea73f-a8e6-46d1-af1b-9228ab5e7dad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:09.802618 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaea73f-a8e6-46d1-af1b-9228ab5e7dad service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:09.802905 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaea73f-a8e6-46d1-af1b-9228ab5e7dad service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:09.803183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaea73f-a8e6-46d1-af1b-9228ab5e7dad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:09.812976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaea73f-a8e6-46d1-af1b-9228ab5e7dad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:09.817118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaea73f-a8e6-46d1-af1b-9228ab5e7dad service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:09.823829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1230/2430] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:47:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:15.028979 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e130a1-75ac-4006-8466-34bdfa272d62 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:15.030819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e130a1-75ac-4006-8466-34bdfa272d62 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:15.031308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e130a1-75ac-4006-8466-34bdfa272d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:15.060829 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f1e130a1-75ac-4006-8466-34bdfa272d62 None None] MFA Rules not processed for user `7a5fc191f3464ffe83d5fde2419cb8b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:15.062200 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f1e130a1-75ac-4006-8466-34bdfa272d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:15.131713 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1201/2431] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:47:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:15.147143 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaea73f-a8e6-46d1-af1b-9228ab5e7dad service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:15.148607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-adaea73f-a8e6-46d1-af1b-9228ab5e7dad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:15.224557 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaea73f-a8e6-46d1-af1b-9228ab5e7dad service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:15.229883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70919ff-525d-404b-b32c-f0f6176e74b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:15.230528 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70919ff-525d-404b-b32c-f0f6176e74b0 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:15.230812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70919ff-525d-404b-b32c-f0f6176e74b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:15.231087 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70919ff-525d-404b-b32c-f0f6176e74b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:15.239012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70919ff-525d-404b-b32c-f0f6176e74b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:15.240580 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70919ff-525d-404b-b32c-f0f6176e74b0 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:15.249410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1231/2432] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:47:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:17.723763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08381530-b0c2-4f7f-a22b-0cd371feb912 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:17.723763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08381530-b0c2-4f7f-a22b-0cd371feb912 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:17.723763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08381530-b0c2-4f7f-a22b-0cd371feb912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:17.739351 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-08381530-b0c2-4f7f-a22b-0cd371feb912 None None] MFA Rules not processed for user `7a5fc191f3464ffe83d5fde2419cb8b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:17.741197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-08381530-b0c2-4f7f-a22b-0cd371feb912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:17.751059 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1202/2433] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:47:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:19.390603 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70919ff-525d-404b-b32c-f0f6176e74b0 service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:19.392204 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a70919ff-525d-404b-b32c-f0f6176e74b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:19.407339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70919ff-525d-404b-b32c-f0f6176e74b0 service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:19.413183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cfedb-b8cf-4f00-85bc-75c62b70bfcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:19.421563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cfedb-b8cf-4f00-85bc-75c62b70bfcb service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:19.421563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cfedb-b8cf-4f00-85bc-75c62b70bfcb service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:19.421563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cfedb-b8cf-4f00-85bc-75c62b70bfcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:19.421563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b25cfedb-b8cf-4f00-85bc-75c62b70bfcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:19.428884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cfedb-b8cf-4f00-85bc-75c62b70bfcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:19.430486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cfedb-b8cf-4f00-85bc-75c62b70bfcb service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:19.441854 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1232/2434] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:47:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:47:23.791232 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08381530-b0c2-4f7f-a22b-0cd371feb912 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:23.806794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08381530-b0c2-4f7f-a22b-0cd371feb912 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:23.814180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d02d1a-4698-4a4a-b77f-7b644b0af37f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:23.814849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d02d1a-4698-4a4a-b77f-7b644b0af37f service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:23.815137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d02d1a-4698-4a4a-b77f-7b644b0af37f service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:23.815415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d02d1a-4698-4a4a-b77f-7b644b0af37f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:23.817378 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d0d02d1a-4698-4a4a-b77f-7b644b0af37f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:23.831961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d02d1a-4698-4a4a-b77f-7b644b0af37f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:23.833674 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d02d1a-4698-4a4a-b77f-7b644b0af37f service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:23.846326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1203/2435] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:47:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:25.097902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cfedb-b8cf-4f00-85bc-75c62b70bfcb service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:25.110075 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cfedb-b8cf-4f00-85bc-75c62b70bfcb service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:25.115664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a162f-3127-46f6-b6ef-cdc460cc379c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:25.116292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a162f-3127-46f6-b6ef-cdc460cc379c service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:25.116579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a162f-3127-46f6-b6ef-cdc460cc379c service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:25.116857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a162f-3127-46f6-b6ef-cdc460cc379c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:25.118635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-688a162f-3127-46f6-b6ef-cdc460cc379c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:25.146347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:25.178691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a162f-3127-46f6-b6ef-cdc460cc379c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:25.180281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a162f-3127-46f6-b6ef-cdc460cc379c service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:25.190605 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1233/2436] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:47:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:33.840460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=55 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:34.215340 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d02d1a-4698-4a4a-b77f-7b644b0af37f service nova] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:34.227455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d02d1a-4698-4a4a-b77f-7b644b0af37f service nova] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:34.233864 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe2373-25f7-4007-a566-5b4456d28904 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:34.234528 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe2373-25f7-4007-a566-5b4456d28904 service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:34.234819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe2373-25f7-4007-a566-5b4456d28904 service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:34.235101 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe2373-25f7-4007-a566-5b4456d28904 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:34.236947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2dbe2373-25f7-4007-a566-5b4456d28904 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:34.250965 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe2373-25f7-4007-a566-5b4456d28904 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:34.252578 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe2373-25f7-4007-a566-5b4456d28904 service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:34.276066 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1204/2437] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:47:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:35.205804 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=15 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:44.259112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=6 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:04.891951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25790e-2fd4-4c54-83dd-5b9cdfeadefb None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:04.892611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25790e-2fd4-4c54-83dd-5b9cdfeadefb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:04.892611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25790e-2fd4-4c54-83dd-5b9cdfeadefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:04.909801 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3f25790e-2fd4-4c54-83dd-5b9cdfeadefb None None] MFA Rules not processed for user `472049224bbf48699b8a9410ec61aa9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:04.910994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3f25790e-2fd4-4c54-83dd-5b9cdfeadefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:04.963503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1234/2438] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:05.314622 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf034fd-79cc-4439-a592-3ebda0597420 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:05.314622 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf034fd-79cc-4439-a592-3ebda0597420 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:05.314622 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf034fd-79cc-4439-a592-3ebda0597420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:05.333791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-abf034fd-79cc-4439-a592-3ebda0597420 None None] MFA Rules not processed for user `ba0229c6732541b4a6dda9040c038686`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:05.335111 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-abf034fd-79cc-4439-a592-3ebda0597420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:05.410338 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1205/2439] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:48:05.556631 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25790e-2fd4-4c54-83dd-5b9cdfeadefb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:05.573477 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25790e-2fd4-4c54-83dd-5b9cdfeadefb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:05.577482 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e909dee-2b47-4d46-8c9a-9886c3eb4d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:05.578148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e909dee-2b47-4d46-8c9a-9886c3eb4d30 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:05.581669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e909dee-2b47-4d46-8c9a-9886c3eb4d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:05.581669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e909dee-2b47-4d46-8c9a-9886c3eb4d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:05.581669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3e909dee-2b47-4d46-8c9a-9886c3eb4d30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:05.591806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e909dee-2b47-4d46-8c9a-9886c3eb4d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:05.593286 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e909dee-2b47-4d46-8c9a-9886c3eb4d30 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:05.604019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1235/2440] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:10.084224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf034fd-79cc-4439-a592-3ebda0597420 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:10.085881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-abf034fd-79cc-4439-a592-3ebda0597420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:10.103631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf034fd-79cc-4439-a592-3ebda0597420 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:10.128738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5751baf-bb5e-4b77-802e-69332b32a012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:10.129654 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5751baf-bb5e-4b77-802e-69332b32a012 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:10.129959 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5751baf-bb5e-4b77-802e-69332b32a012 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:10.130260 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5751baf-bb5e-4b77-802e-69332b32a012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:10.133795 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f5751baf-bb5e-4b77-802e-69332b32a012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:10.153493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5751baf-bb5e-4b77-802e-69332b32a012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:10.153493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5751baf-bb5e-4b77-802e-69332b32a012 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:10.167800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1206/2441] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:10.296622 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e909dee-2b47-4d46-8c9a-9886c3eb4d30 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:10.308362 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e909dee-2b47-4d46-8c9a-9886c3eb4d30 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:10.319299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3faffb9-ba02-41f9-ac1a-69c6b9d953de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:10.322151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3faffb9-ba02-41f9-ac1a-69c6b9d953de service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:10.322151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3faffb9-ba02-41f9-ac1a-69c6b9d953de service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:10.322151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3faffb9-ba02-41f9-ac1a-69c6b9d953de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:10.324653 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f3faffb9-ba02-41f9-ac1a-69c6b9d953de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:10.341623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3faffb9-ba02-41f9-ac1a-69c6b9d953de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:10.343986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3faffb9-ba02-41f9-ac1a-69c6b9d953de service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:10.362222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1236/2442] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:14.320459 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5751baf-bb5e-4b77-802e-69332b32a012 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:14.321969 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f5751baf-bb5e-4b77-802e-69332b32a012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:14.335592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5751baf-bb5e-4b77-802e-69332b32a012 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:14.340138 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae711ddc-8dc8-4582-b9a7-93e213c7cada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:14.341093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae711ddc-8dc8-4582-b9a7-93e213c7cada admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:14.341565 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae711ddc-8dc8-4582-b9a7-93e213c7cada admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:14.342008 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae711ddc-8dc8-4582-b9a7-93e213c7cada admin admin] PATH_INFO: `/v3/users/5236a93fc3a84d27bf3d150e85eb2589` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:14.344039 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae711ddc-8dc8-4582-b9a7-93e213c7cada admin admin] RBAC: Authorizing `identity:delete_user(user_id=5236a93fc3a84d27bf3d150e85eb2589)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:14.347156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae711ddc-8dc8-4582-b9a7-93e213c7cada admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:14.626037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ae711ddc-8dc8-4582-b9a7-93e213c7cada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5236a93fc3a84d27bf3d150e85eb2589'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:14.627309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ae711ddc-8dc8-4582-b9a7-93e213c7cada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5236a93fc3a84d27bf3d150e85eb2589'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:14.627921 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ae711ddc-8dc8-4582-b9a7-93e213c7cada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5236a93fc3a84d27bf3d150e85eb2589'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:14.634721 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ae711ddc-8dc8-4582-b9a7-93e213c7cada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5236a93fc3a84d27bf3d150e85eb2589'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:14.646846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-ae711ddc-8dc8-4582-b9a7-93e213c7cada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5236a93fc3a84d27bf3d150e85eb2589'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:14.754168 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1207/2443] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:48:14 2022] DELETE /identity/v3/users/5236a93fc3a84d27bf3d150e85eb2589 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:14.995452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3faffb9-ba02-41f9-ac1a-69c6b9d953de service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:15.007056 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3faffb9-ba02-41f9-ac1a-69c6b9d953de service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:15.068632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e8e74-2744-4635-8680-e0e17ced849e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:15.069525 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e8e74-2744-4635-8680-e0e17ced849e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:15.069814 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e8e74-2744-4635-8680-e0e17ced849e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:15.070094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e8e74-2744-4635-8680-e0e17ced849e admin admin] PATH_INFO: `/v3/projects/8d7ac236cd4d414fb70eb8da6da53484` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:15.071265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e8e74-2744-4635-8680-e0e17ced849e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d7ac236cd4d414fb70eb8da6da53484)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:15.073207 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e8e74-2744-4635-8680-e0e17ced849e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:15.195498 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0e2e8e74-2744-4635-8680-e0e17ced849e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d7ac236cd4d414fb70eb8da6da53484', 'request_id': 'req-0e2e8e74-2744-4635-8680-e0e17ced849e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:15.201877 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0e2e8e74-2744-4635-8680-e0e17ced849e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d7ac236cd4d414fb70eb8da6da53484', 'request_id': 'req-0e2e8e74-2744-4635-8680-e0e17ced849e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:15.290960 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0e2e8e74-2744-4635-8680-e0e17ced849e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d7ac236cd4d414fb70eb8da6da53484', 'request_id': 'req-0e2e8e74-2744-4635-8680-e0e17ced849e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:15.304600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0e2e8e74-2744-4635-8680-e0e17ced849e admin admin] The token cache is being invalidate because project 8d7ac236cd4d414fb70eb8da6da53484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:48:15.304895 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0e2e8e74-2744-4635-8680-e0e17ced849e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:15.305566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0e2e8e74-2744-4635-8680-e0e17ced849e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:15.307123 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1237/2444] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:48:14 2022] DELETE /identity/v3/projects/8d7ac236cd4d414fb70eb8da6da53484 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:48:15.312149 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee257c4e-d150-44a0-b123-6aa640416e10 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:15.318520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee257c4e-d150-44a0-b123-6aa640416e10 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:15.318520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee257c4e-d150-44a0-b123-6aa640416e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:15.333481 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ee257c4e-d150-44a0-b123-6aa640416e10 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:15.335471 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ee257c4e-d150-44a0-b123-6aa640416e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:15.401439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1208/2445] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:15.409999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84602df2-2f20-4a9c-b9e3-30d832b7dd3c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:15.410265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84602df2-2f20-4a9c-b9e3-30d832b7dd3c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:15.410516 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84602df2-2f20-4a9c-b9e3-30d832b7dd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:15.431587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-84602df2-2f20-4a9c-b9e3-30d832b7dd3c None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:15.432851 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-84602df2-2f20-4a9c-b9e3-30d832b7dd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:15.442768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1238/2446] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:15.447054 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee257c4e-d150-44a0-b123-6aa640416e10 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:15.453682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:15.457743 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee257c4e-d150-44a0-b123-6aa640416e10 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:15.462520 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe19fb-8276-4a06-935c-6f782e9455fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:15.470060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe19fb-8276-4a06-935c-6f782e9455fc admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:15.470060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe19fb-8276-4a06-935c-6f782e9455fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:15.470060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe19fb-8276-4a06-935c-6f782e9455fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:15.470060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe19fb-8276-4a06-935c-6f782e9455fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:48:15.470060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe19fb-8276-4a06-935c-6f782e9455fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:15.470060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe19fb-8276-4a06-935c-6f782e9455fc admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:15.478787 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1209/2447] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:48:15.483040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84602df2-2f20-4a9c-b9e3-30d832b7dd3c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:15.488607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:15.493898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84602df2-2f20-4a9c-b9e3-30d832b7dd3c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:15.498643 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9a4fe-50db-4a83-8abc-f09a8778e6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:15.499774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9a4fe-50db-4a83-8abc-f09a8778e6bd admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:15.499774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9a4fe-50db-4a83-8abc-f09a8778e6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:15.499943 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9a4fe-50db-4a83-8abc-f09a8778e6bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:15.500536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9a4fe-50db-4a83-8abc-f09a8778e6bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:15.503126 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9a4fe-50db-4a83-8abc-f09a8778e6bd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:15.617672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1239/2448] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:48:15 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:48:15.619998 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe19fb-8276-4a06-935c-6f782e9455fc admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:15.636274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe19fb-8276-4a06-935c-6f782e9455fc admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:15.719403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aff8c-972f-4b73-a624-832edb3467cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:15.721264 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aff8c-972f-4b73-a624-832edb3467cc admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:15.721264 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aff8c-972f-4b73-a624-832edb3467cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:15.721667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aff8c-972f-4b73-a624-832edb3467cc admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:15.722469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445aff8c-972f-4b73-a624-832edb3467cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:15.724494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445aff8c-972f-4b73-a624-832edb3467cc admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:15.844566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1210/2449] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:48:15 2022] POST /identity/v3/users => generated 294 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:48:15.846862 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9a4fe-50db-4a83-8abc-f09a8778e6bd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:15.857254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9a4fe-50db-4a83-8abc-f09a8778e6bd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:15.862143 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc21c4-135b-4906-902d-52902227222d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:15.862821 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc21c4-135b-4906-902d-52902227222d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:15.863106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc21c4-135b-4906-902d-52902227222d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:15.863384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc21c4-135b-4906-902d-52902227222d admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:15.864118 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc21c4-135b-4906-902d-52902227222d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:15.867785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc21c4-135b-4906-902d-52902227222d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:15.882404 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1240/2450] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:48:15.891083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aff8c-972f-4b73-a624-832edb3467cc admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:15.906577 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aff8c-972f-4b73-a624-832edb3467cc admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:15.913298 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6eaa9d-d8c2-4070-868d-7b6c06c62772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:15.914007 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6eaa9d-d8c2-4070-868d-7b6c06c62772 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:15.914298 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6eaa9d-d8c2-4070-868d-7b6c06c62772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:15.914635 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6eaa9d-d8c2-4070-868d-7b6c06c62772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:15.915512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6eaa9d-d8c2-4070-868d-7b6c06c62772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:15.917184 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6eaa9d-d8c2-4070-868d-7b6c06c62772 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:15.928723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1211/2451] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:48:15.932746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc21c4-135b-4906-902d-52902227222d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:15.943368 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc21c4-135b-4906-902d-52902227222d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:15.947972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32692960-3a12-4175-b4b3-11150b02640e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:15.948857 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32692960-3a12-4175-b4b3-11150b02640e admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:15.949170 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32692960-3a12-4175-b4b3-11150b02640e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:15.949455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32692960-3a12-4175-b4b3-11150b02640e admin admin] PATH_INFO: `/v3/projects/2d6864fc29d5485bb70736085a9fcf41/users/0685e1deaea44a6db5e6f8df6e26b6a7/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:15.966352 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32692960-3a12-4175-b4b3-11150b02640e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6864fc29d5485bb70736085a9fcf41, user_id=0685e1deaea44a6db5e6f8df6e26b6a7, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:15.969120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32692960-3a12-4175-b4b3-11150b02640e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:16.045538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1241/2452] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:48:15 2022] PUT /identity/v3/projects/2d6864fc29d5485bb70736085a9fcf41/users/0685e1deaea44a6db5e6f8df6e26b6a7/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:16.057301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1efb1-a202-4ec3-a890-f487aa4a41db None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:16.057601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1efb1-a202-4ec3-a890-f487aa4a41db None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:16.057887 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1efb1-a202-4ec3-a890-f487aa4a41db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:16.086112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e7c1efb1-a202-4ec3-a890-f487aa4a41db None None] MFA Rules not processed for user `0685e1deaea44a6db5e6f8df6e26b6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:16.087345 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e7c1efb1-a202-4ec3-a890-f487aa4a41db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:16.154095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1212/2453] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:16.158861 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32692960-3a12-4175-b4b3-11150b02640e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:16.232716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32692960-3a12-4175-b4b3-11150b02640e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:16.237490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc581f98-4c66-47c4-964e-551b1a500b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:16.238159 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc581f98-4c66-47c4-964e-551b1a500b0a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:16.238449 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc581f98-4c66-47c4-964e-551b1a500b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:16.238731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc581f98-4c66-47c4-964e-551b1a500b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:16.239437 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc581f98-4c66-47c4-964e-551b1a500b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:16.241376 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc581f98-4c66-47c4-964e-551b1a500b0a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:16.352130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1242/2454] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:48:16 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:48:16.355982 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1efb1-a202-4ec3-a890-f487aa4a41db None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:16.369327 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1efb1-a202-4ec3-a890-f487aa4a41db None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:16.426506 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f74f6c-9698-4311-b8b1-e57745a8448c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:16.427184 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f74f6c-9698-4311-b8b1-e57745a8448c admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:16.427503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f74f6c-9698-4311-b8b1-e57745a8448c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:16.427786 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f74f6c-9698-4311-b8b1-e57745a8448c admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:16.428473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f74f6c-9698-4311-b8b1-e57745a8448c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:16.430456 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f74f6c-9698-4311-b8b1-e57745a8448c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:16.437099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:16.548195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1213/2455] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:48:16 2022] POST /identity/v3/users => generated 299 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:48:16.552842 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc581f98-4c66-47c4-964e-551b1a500b0a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:16.563202 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc581f98-4c66-47c4-964e-551b1a500b0a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:16.567897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548865d2-b46a-4704-a6db-cfa941f0c474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:16.568546 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548865d2-b46a-4704-a6db-cfa941f0c474 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:16.568831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548865d2-b46a-4704-a6db-cfa941f0c474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:16.569224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548865d2-b46a-4704-a6db-cfa941f0c474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:16.569899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548865d2-b46a-4704-a6db-cfa941f0c474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:16.571538 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548865d2-b46a-4704-a6db-cfa941f0c474 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:16.583183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1243/2456] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:48:16.587498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f74f6c-9698-4311-b8b1-e57745a8448c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:16.598089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f74f6c-9698-4311-b8b1-e57745a8448c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:16.602561 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83520f30-3aa7-4fda-9c0a-45d8b42ef853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:16.603430 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83520f30-3aa7-4fda-9c0a-45d8b42ef853 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:16.603712 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83520f30-3aa7-4fda-9c0a-45d8b42ef853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:16.603990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83520f30-3aa7-4fda-9c0a-45d8b42ef853 admin admin] PATH_INFO: `/v3/projects/6210406b1c6944d3b4efe0245676b23e/users/3cd0c7e623474123bdd358c0b282739b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:16.618489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83520f30-3aa7-4fda-9c0a-45d8b42ef853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6210406b1c6944d3b4efe0245676b23e, user_id=3cd0c7e623474123bdd358c0b282739b, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:16.621182 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83520f30-3aa7-4fda-9c0a-45d8b42ef853 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:16.691389 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1214/2457] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:48:16 2022] PUT /identity/v3/projects/6210406b1c6944d3b4efe0245676b23e/users/3cd0c7e623474123bdd358c0b282739b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:48:16.695358 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548865d2-b46a-4704-a6db-cfa941f0c474 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:16.706312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548865d2-b46a-4704-a6db-cfa941f0c474 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:16.761396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd46c5a-fe87-4b01-aed1-efdc4f0e7ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:16.762068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd46c5a-fe87-4b01-aed1-efdc4f0e7ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:16.762359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd46c5a-fe87-4b01-aed1-efdc4f0e7ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:16.762638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd46c5a-fe87-4b01-aed1-efdc4f0e7ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:16.763364 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd46c5a-fe87-4b01-aed1-efdc4f0e7ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:16.764950 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd46c5a-fe87-4b01-aed1-efdc4f0e7ff3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:16.775431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1244/2458] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:48:16.779618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83520f30-3aa7-4fda-9c0a-45d8b42ef853 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:16.790507 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83520f30-3aa7-4fda-9c0a-45d8b42ef853 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:16.795089 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fb8eb-f2a4-4ba3-a604-745637fc508a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:16.795957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fb8eb-f2a4-4ba3-a604-745637fc508a admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:16.796244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fb8eb-f2a4-4ba3-a604-745637fc508a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:16.796532 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fb8eb-f2a4-4ba3-a604-745637fc508a admin admin] PATH_INFO: `/v3/projects/6210406b1c6944d3b4efe0245676b23e/users/3cd0c7e623474123bdd358c0b282739b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:16.798619 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384fb8eb-f2a4-4ba3-a604-745637fc508a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6210406b1c6944d3b4efe0245676b23e, user_id=3cd0c7e623474123bdd358c0b282739b, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:16.801557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384fb8eb-f2a4-4ba3-a604-745637fc508a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:16.811558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1215/2459] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:48:16 2022] PUT /identity/v3/projects/6210406b1c6944d3b4efe0245676b23e/users/3cd0c7e623474123bdd358c0b282739b/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:16.821671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25cffc-53e5-498c-9343-cc8c66c9b855 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:16.821968 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25cffc-53e5-498c-9343-cc8c66c9b855 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:16.822251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25cffc-53e5-498c-9343-cc8c66c9b855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:16.835290 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:16.850117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ac25cffc-53e5-498c-9343-cc8c66c9b855 None None] MFA Rules not processed for user `3cd0c7e623474123bdd358c0b282739b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:16.851342 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ac25cffc-53e5-498c-9343-cc8c66c9b855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:16.921008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1245/2460] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:16.938374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fb8eb-f2a4-4ba3-a604-745637fc508a admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:16.941189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-384fb8eb-f2a4-4ba3-a604-745637fc508a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:17.006778 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fb8eb-f2a4-4ba3-a604-745637fc508a admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:17.010548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7642a25-70a7-4954-a518-994b1387b971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:17.011202 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7642a25-70a7-4954-a518-994b1387b971 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:17.011517 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7642a25-70a7-4954-a518-994b1387b971 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:17.011858 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7642a25-70a7-4954-a518-994b1387b971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:17.022785 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7642a25-70a7-4954-a518-994b1387b971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:17.024236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7642a25-70a7-4954-a518-994b1387b971 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:17.033980 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1216/2461] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:17.062852 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25cffc-53e5-498c-9343-cc8c66c9b855 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:17.126038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25cffc-53e5-498c-9343-cc8c66c9b855 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:17.131076 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c84046-da0b-4a51-9291-9716a1dfd2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:17.131956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c84046-da0b-4a51-9291-9716a1dfd2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:17.132308 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c84046-da0b-4a51-9291-9716a1dfd2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:17.132513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c84046-da0b-4a51-9291-9716a1dfd2fe admin admin] PATH_INFO: `/v3/users/0685e1deaea44a6db5e6f8df6e26b6a7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:17.133744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c84046-da0b-4a51-9291-9716a1dfd2fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=0685e1deaea44a6db5e6f8df6e26b6a7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:17.135683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c84046-da0b-4a51-9291-9716a1dfd2fe admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:17.315607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-95c84046-da0b-4a51-9291-9716a1dfd2fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0685e1deaea44a6db5e6f8df6e26b6a7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:17.327363 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-95c84046-da0b-4a51-9291-9716a1dfd2fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0685e1deaea44a6db5e6f8df6e26b6a7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:17.388132 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-95c84046-da0b-4a51-9291-9716a1dfd2fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0685e1deaea44a6db5e6f8df6e26b6a7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:17.388748 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-95c84046-da0b-4a51-9291-9716a1dfd2fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0685e1deaea44a6db5e6f8df6e26b6a7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:17.396228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-95c84046-da0b-4a51-9291-9716a1dfd2fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0685e1deaea44a6db5e6f8df6e26b6a7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:17.402176 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1246/2462] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:48:17 2022] DELETE /identity/v3/users/0685e1deaea44a6db5e6f8df6e26b6a7 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:17.406754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37e9f9-0e53-4eb3-95b3-9fd2b2332b74 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:17.407203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37e9f9-0e53-4eb3-95b3-9fd2b2332b74 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:17.407328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37e9f9-0e53-4eb3-95b3-9fd2b2332b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:17.425600 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2f37e9f9-0e53-4eb3-95b3-9fd2b2332b74 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:17.426893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2f37e9f9-0e53-4eb3-95b3-9fd2b2332b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:17.483341 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1217/2463] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:17.493502 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c84046-da0b-4a51-9291-9716a1dfd2fe admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:17.494995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-95c84046-da0b-4a51-9291-9716a1dfd2fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:17.508817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:17.551362 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c84046-da0b-4a51-9291-9716a1dfd2fe admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:17.558436 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca04a4-8c4f-4ef9-b311-08d688da818f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:17.558436 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca04a4-8c4f-4ef9-b311-08d688da818f service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:17.558436 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca04a4-8c4f-4ef9-b311-08d688da818f service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:17.558436 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca04a4-8c4f-4ef9-b311-08d688da818f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:17.565973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca04a4-8c4f-4ef9-b311-08d688da818f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:17.567344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca04a4-8c4f-4ef9-b311-08d688da818f service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:17.576616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1247/2464] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:18.018895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37e9f9-0e53-4eb3-95b3-9fd2b2332b74 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:18.021008 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2f37e9f9-0e53-4eb3-95b3-9fd2b2332b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:18.037771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37e9f9-0e53-4eb3-95b3-9fd2b2332b74 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:18.040417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56af8c-c132-4290-b6ff-6d656eb7e815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:18.041410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56af8c-c132-4290-b6ff-6d656eb7e815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:18.041817 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56af8c-c132-4290-b6ff-6d656eb7e815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:18.042100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56af8c-c132-4290-b6ff-6d656eb7e815 admin admin] PATH_INFO: `/v3/projects/2d6864fc29d5485bb70736085a9fcf41` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:18.043334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56af8c-c132-4290-b6ff-6d656eb7e815 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d6864fc29d5485bb70736085a9fcf41)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:18.046715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56af8c-c132-4290-b6ff-6d656eb7e815 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:18.176133 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-af56af8c-c132-4290-b6ff-6d656eb7e815 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d6864fc29d5485bb70736085a9fcf41', 'request_id': 'req-af56af8c-c132-4290-b6ff-6d656eb7e815'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:18.178632 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:18.184016 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-af56af8c-c132-4290-b6ff-6d656eb7e815 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d6864fc29d5485bb70736085a9fcf41', 'request_id': 'req-af56af8c-c132-4290-b6ff-6d656eb7e815'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:18.190565 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-af56af8c-c132-4290-b6ff-6d656eb7e815 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d6864fc29d5485bb70736085a9fcf41', 'request_id': 'req-af56af8c-c132-4290-b6ff-6d656eb7e815'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:18.315875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-af56af8c-c132-4290-b6ff-6d656eb7e815 admin admin] The token cache is being invalidate because project 2d6864fc29d5485bb70736085a9fcf41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:48:18.316586 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-af56af8c-c132-4290-b6ff-6d656eb7e815 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:18.317465 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-af56af8c-c132-4290-b6ff-6d656eb7e815 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:18.320408 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1218/2465] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:48:18 2022] DELETE /identity/v3/projects/2d6864fc29d5485bb70736085a9fcf41 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:48:18.324146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca04a4-8c4f-4ef9-b311-08d688da818f service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:18.334525 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca04a4-8c4f-4ef9-b311-08d688da818f service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:18.396843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9736b-134b-4d33-953a-7a8b0380ad67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:18.397783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9736b-134b-4d33-953a-7a8b0380ad67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:18.398077 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9736b-134b-4d33-953a-7a8b0380ad67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:18.398353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9736b-134b-4d33-953a-7a8b0380ad67 admin admin] PATH_INFO: `/v3/users/3cd0c7e623474123bdd358c0b282739b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:18.399544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9736b-134b-4d33-953a-7a8b0380ad67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cd0c7e623474123bdd358c0b282739b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:18.401519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9736b-134b-4d33-953a-7a8b0380ad67 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:18.676203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b1e9736b-134b-4d33-953a-7a8b0380ad67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cd0c7e623474123bdd358c0b282739b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:18.687999 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b1e9736b-134b-4d33-953a-7a8b0380ad67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cd0c7e623474123bdd358c0b282739b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:18.804912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b1e9736b-134b-4d33-953a-7a8b0380ad67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cd0c7e623474123bdd358c0b282739b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:18.805578 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b1e9736b-134b-4d33-953a-7a8b0380ad67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cd0c7e623474123bdd358c0b282739b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:18.813152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b1e9736b-134b-4d33-953a-7a8b0380ad67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cd0c7e623474123bdd358c0b282739b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:18.819697 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1248/2466] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:48:18 2022] DELETE /identity/v3/users/3cd0c7e623474123bdd358c0b282739b => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:19.263736 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56af8c-c132-4290-b6ff-6d656eb7e815 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:19.265358 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-af56af8c-c132-4290-b6ff-6d656eb7e815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:19.279273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56af8c-c132-4290-b6ff-6d656eb7e815 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:19.339728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf77560-e3a6-4210-98f6-18955cd7bd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:19.340586 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf77560-e3a6-4210-98f6-18955cd7bd30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:19.340875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf77560-e3a6-4210-98f6-18955cd7bd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:19.341188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf77560-e3a6-4210-98f6-18955cd7bd30 admin admin] PATH_INFO: `/v3/projects/6210406b1c6944d3b4efe0245676b23e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:19.342375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf77560-e3a6-4210-98f6-18955cd7bd30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6210406b1c6944d3b4efe0245676b23e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:19.344312 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf77560-e3a6-4210-98f6-18955cd7bd30 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:19.463774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6cf77560-e3a6-4210-98f6-18955cd7bd30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6210406b1c6944d3b4efe0245676b23e', 'request_id': 'req-6cf77560-e3a6-4210-98f6-18955cd7bd30'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:19.471445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6cf77560-e3a6-4210-98f6-18955cd7bd30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6210406b1c6944d3b4efe0245676b23e', 'request_id': 'req-6cf77560-e3a6-4210-98f6-18955cd7bd30'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:19.482762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6cf77560-e3a6-4210-98f6-18955cd7bd30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6210406b1c6944d3b4efe0245676b23e', 'request_id': 'req-6cf77560-e3a6-4210-98f6-18955cd7bd30'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:19.719349 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6cf77560-e3a6-4210-98f6-18955cd7bd30 admin admin] The token cache is being invalidate because project 6210406b1c6944d3b4efe0245676b23e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:48:19.722716 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6cf77560-e3a6-4210-98f6-18955cd7bd30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:19.722716 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6cf77560-e3a6-4210-98f6-18955cd7bd30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:19.723077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1219/2467] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:48:19 2022] DELETE /identity/v3/projects/6210406b1c6944d3b4efe0245676b23e => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:19.728508 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb607b-bb86-43e9-8da1-b65d4b0308dc None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:19.729001 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb607b-bb86-43e9-8da1-b65d4b0308dc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:19.729001 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb607b-bb86-43e9-8da1-b65d4b0308dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:19.747893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-14eb607b-bb86-43e9-8da1-b65d4b0308dc None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:19.749221 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-14eb607b-bb86-43e9-8da1-b65d4b0308dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:19.780445 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:19.811399 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1249/2468] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:48:19.819302 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bbaf5-2869-46a8-98d6-a63b56020beb None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:19.819538 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bbaf5-2869-46a8-98d6-a63b56020beb None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:19.819824 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bbaf5-2869-46a8-98d6-a63b56020beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:19.838500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7f7bbaf5-2869-46a8-98d6-a63b56020beb None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:19.839738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7f7bbaf5-2869-46a8-98d6-a63b56020beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:19.848719 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1220/2469] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:19.852880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb607b-bb86-43e9-8da1-b65d4b0308dc None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:19.863327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb607b-bb86-43e9-8da1-b65d4b0308dc None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:19.867899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeaa245-7ceb-461e-8a50-4c5e0f8528f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:19.868551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaa245-7ceb-461e-8a50-4c5e0f8528f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:19.868839 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaa245-7ceb-461e-8a50-4c5e0f8528f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:19.869149 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaa245-7ceb-461e-8a50-4c5e0f8528f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:19.869875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeaa245-7ceb-461e-8a50-4c5e0f8528f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:48:19.870181 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeaa245-7ceb-461e-8a50-4c5e0f8528f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:19.871773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeaa245-7ceb-461e-8a50-4c5e0f8528f6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:19.886022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1250/2470] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:48:19.889994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7bbaf5-2869-46a8-98d6-a63b56020beb None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:19.900580 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7bbaf5-2869-46a8-98d6-a63b56020beb None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:19.905346 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a445a3-6cc0-4f06-aa40-8cc8f29e85d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:19.906009 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a445a3-6cc0-4f06-aa40-8cc8f29e85d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:19.906293 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a445a3-6cc0-4f06-aa40-8cc8f29e85d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:19.906571 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a445a3-6cc0-4f06-aa40-8cc8f29e85d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:19.907263 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a445a3-6cc0-4f06-aa40-8cc8f29e85d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:19.909586 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a445a3-6cc0-4f06-aa40-8cc8f29e85d1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:20.014116 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1221/2471] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:48:19 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:48:20.018339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeaa245-7ceb-461e-8a50-4c5e0f8528f6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:20.028402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeaa245-7ceb-461e-8a50-4c5e0f8528f6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:20.088849 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bec4d-3197-4745-82f7-93fd8f3108f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:20.089424 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bec4d-3197-4745-82f7-93fd8f3108f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:20.090046 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bec4d-3197-4745-82f7-93fd8f3108f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:20.090046 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bec4d-3197-4745-82f7-93fd8f3108f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:20.090684 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35bec4d-3197-4745-82f7-93fd8f3108f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:20.092639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35bec4d-3197-4745-82f7-93fd8f3108f9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:20.171111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1251/2472] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:48:20 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:48:20.176063 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a445a3-6cc0-4f06-aa40-8cc8f29e85d1 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:20.186457 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a445a3-6cc0-4f06-aa40-8cc8f29e85d1 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:20.191696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42f352-c67b-4b94-a2df-dbda0eab5d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:20.192395 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42f352-c67b-4b94-a2df-dbda0eab5d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:20.192683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42f352-c67b-4b94-a2df-dbda0eab5d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:20.192963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42f352-c67b-4b94-a2df-dbda0eab5d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:20.193731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42f352-c67b-4b94-a2df-dbda0eab5d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:20.195324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42f352-c67b-4b94-a2df-dbda0eab5d73 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:20.206868 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1222/2473] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:48:20.211024 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bec4d-3197-4745-82f7-93fd8f3108f9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:20.221740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bec4d-3197-4745-82f7-93fd8f3108f9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:20.226529 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdb4ec-3e5c-42b7-98cf-b4843fbcdc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:20.227196 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdb4ec-3e5c-42b7-98cf-b4843fbcdc4f admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:20.227483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdb4ec-3e5c-42b7-98cf-b4843fbcdc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:20.227764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdb4ec-3e5c-42b7-98cf-b4843fbcdc4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:20.228487 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdb4ec-3e5c-42b7-98cf-b4843fbcdc4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:20.230125 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdb4ec-3e5c-42b7-98cf-b4843fbcdc4f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:20.240145 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1252/2474] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:48:20.244288 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42f352-c67b-4b94-a2df-dbda0eab5d73 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:20.254924 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42f352-c67b-4b94-a2df-dbda0eab5d73 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:20.259651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149cc74-9cfc-4f4f-899a-f16e9b3e9c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:20.260531 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149cc74-9cfc-4f4f-899a-f16e9b3e9c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:20.260823 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149cc74-9cfc-4f4f-899a-f16e9b3e9c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:20.261135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149cc74-9cfc-4f4f-899a-f16e9b3e9c4a admin admin] PATH_INFO: `/v3/projects/060da834e0574299a5712508e54d7ba7/users/13aa318a9a37496385e7df324c6a1c50/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:20.276369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c149cc74-9cfc-4f4f-899a-f16e9b3e9c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=060da834e0574299a5712508e54d7ba7, user_id=13aa318a9a37496385e7df324c6a1c50, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:20.279181 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c149cc74-9cfc-4f4f-899a-f16e9b3e9c4a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:20.360126 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1223/2475] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:48:20 2022] PUT /identity/v3/projects/060da834e0574299a5712508e54d7ba7/users/13aa318a9a37496385e7df324c6a1c50/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:48:20.372039 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ef98a-e0ee-4864-8f51-957307985e7f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:20.373571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ef98a-e0ee-4864-8f51-957307985e7f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:20.373571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ef98a-e0ee-4864-8f51-957307985e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:20.402300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-366ef98a-e0ee-4864-8f51-957307985e7f None None] MFA Rules not processed for user `13aa318a9a37496385e7df324c6a1c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:20.403522 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-366ef98a-e0ee-4864-8f51-957307985e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:20.472878 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1253/2476] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:20.477630 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149cc74-9cfc-4f4f-899a-f16e9b3e9c4a admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:20.484018 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:20.544085 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149cc74-9cfc-4f4f-899a-f16e9b3e9c4a admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:20.549895 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10040384-d6ef-4600-ae54-1472b378febd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:20.550559 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10040384-d6ef-4600-ae54-1472b378febd admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:20.550844 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10040384-d6ef-4600-ae54-1472b378febd admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:20.551120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10040384-d6ef-4600-ae54-1472b378febd admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:20.551807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10040384-d6ef-4600-ae54-1472b378febd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:20.553748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10040384-d6ef-4600-ae54-1472b378febd admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:20.709441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1224/2477] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:48:20 2022] POST /identity/v3/projects => generated 383 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:48:20.713377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ef98a-e0ee-4864-8f51-957307985e7f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:20.725035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ef98a-e0ee-4864-8f51-957307985e7f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:20.751704 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:20.784604 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37638d-5c9f-4832-add9-3181de009ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:20.785415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37638d-5c9f-4832-add9-3181de009ecc admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:20.785709 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37638d-5c9f-4832-add9-3181de009ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:20.785994 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37638d-5c9f-4832-add9-3181de009ecc admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:20.786703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37638d-5c9f-4832-add9-3181de009ecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:20.788700 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37638d-5c9f-4832-add9-3181de009ecc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:20.900459 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1254/2478] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:48:20 2022] POST /identity/v3/users => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:48:20.901432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10040384-d6ef-4600-ae54-1472b378febd admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:20.913187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10040384-d6ef-4600-ae54-1472b378febd admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:20.918069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c318c0-9f72-4862-a17a-a111dbbcff5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:20.918742 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c318c0-9f72-4862-a17a-a111dbbcff5f admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:20.919704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c318c0-9f72-4862-a17a-a111dbbcff5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:20.919704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c318c0-9f72-4862-a17a-a111dbbcff5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:20.920062 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c318c0-9f72-4862-a17a-a111dbbcff5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:20.922113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c318c0-9f72-4862-a17a-a111dbbcff5f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:20.934448 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1225/2479] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:48:20.950275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37638d-5c9f-4832-add9-3181de009ecc admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:20.961162 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37638d-5c9f-4832-add9-3181de009ecc admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:20.966075 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73338f49-e989-4300-9038-5fb3be352816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:20.967038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73338f49-e989-4300-9038-5fb3be352816 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:20.967333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73338f49-e989-4300-9038-5fb3be352816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:20.967615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73338f49-e989-4300-9038-5fb3be352816 admin admin] PATH_INFO: `/v3/projects/102f0d9088174e77bb51473facfedc6c/users/1b5be7e638b34090845fc9ab13aa5c06/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:20.985011 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73338f49-e989-4300-9038-5fb3be352816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=102f0d9088174e77bb51473facfedc6c, user_id=1b5be7e638b34090845fc9ab13aa5c06, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:20.989134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73338f49-e989-4300-9038-5fb3be352816 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:21.081240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1255/2480] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:48:20 2022] PUT /identity/v3/projects/102f0d9088174e77bb51473facfedc6c/users/1b5be7e638b34090845fc9ab13aa5c06/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:21.085343 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c318c0-9f72-4862-a17a-a111dbbcff5f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:21.096410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c318c0-9f72-4862-a17a-a111dbbcff5f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:21.153358 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a91b7-89d7-47b5-b4ff-263de504b53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:21.154323 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a91b7-89d7-47b5-b4ff-263de504b53b admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:21.155257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a91b7-89d7-47b5-b4ff-263de504b53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:21.155541 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a91b7-89d7-47b5-b4ff-263de504b53b admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:21.156270 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a91b7-89d7-47b5-b4ff-263de504b53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:21.158804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a91b7-89d7-47b5-b4ff-263de504b53b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:21.169260 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1226/2481] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:48:21.173821 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73338f49-e989-4300-9038-5fb3be352816 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:21.187620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73338f49-e989-4300-9038-5fb3be352816 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:21.194859 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4206de-32d4-4e22-9991-10ff40da602b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:21.196128 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4206de-32d4-4e22-9991-10ff40da602b admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:21.196423 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4206de-32d4-4e22-9991-10ff40da602b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:21.196798 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4206de-32d4-4e22-9991-10ff40da602b admin admin] PATH_INFO: `/v3/projects/102f0d9088174e77bb51473facfedc6c/users/1b5be7e638b34090845fc9ab13aa5c06/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:21.199106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4206de-32d4-4e22-9991-10ff40da602b admin admin] RBAC: Authorizing `identity:create_grant(project_id=102f0d9088174e77bb51473facfedc6c, user_id=1b5be7e638b34090845fc9ab13aa5c06, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:21.202539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4206de-32d4-4e22-9991-10ff40da602b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:21.215982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1256/2482] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:48:21 2022] PUT /identity/v3/projects/102f0d9088174e77bb51473facfedc6c/users/1b5be7e638b34090845fc9ab13aa5c06/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:21.227601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d1ce2-96d5-452f-a3c2-413994b5a2f4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:21.228043 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d1ce2-96d5-452f-a3c2-413994b5a2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:21.228533 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d1ce2-96d5-452f-a3c2-413994b5a2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:21.258816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6f8d1ce2-96d5-452f-a3c2-413994b5a2f4 None None] MFA Rules not processed for user `1b5be7e638b34090845fc9ab13aa5c06`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:21.260185 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6f8d1ce2-96d5-452f-a3c2-413994b5a2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:21.325763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:21.330123 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1227/2483] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:21.342363 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4206de-32d4-4e22-9991-10ff40da602b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:21.344342 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ab4206de-32d4-4e22-9991-10ff40da602b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:21.404311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4206de-32d4-4e22-9991-10ff40da602b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:21.408220 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19146fa-f316-4547-bceb-ef7945e7dea5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:21.408880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19146fa-f316-4547-bceb-ef7945e7dea5 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:21.409182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19146fa-f316-4547-bceb-ef7945e7dea5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:21.409468 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19146fa-f316-4547-bceb-ef7945e7dea5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:21.418728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19146fa-f316-4547-bceb-ef7945e7dea5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:21.420159 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19146fa-f316-4547-bceb-ef7945e7dea5 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:21.430058 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1257/2484] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:21.534055 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d1ce2-96d5-452f-a3c2-413994b5a2f4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:21.543886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d1ce2-96d5-452f-a3c2-413994b5a2f4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:21.547582 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1759df-c704-470f-8263-8ee239cc3fed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:21.551919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1759df-c704-470f-8263-8ee239cc3fed service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:21.551919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1759df-c704-470f-8263-8ee239cc3fed service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:21.551919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1759df-c704-470f-8263-8ee239cc3fed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:21.608441 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1759df-c704-470f-8263-8ee239cc3fed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:21.610028 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1759df-c704-470f-8263-8ee239cc3fed service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:21.620019 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1228/2485] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:24.798331 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19146fa-f316-4547-bceb-ef7945e7dea5 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:24.870409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19146fa-f316-4547-bceb-ef7945e7dea5 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:24.875054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddc994-8149-40ef-b9ca-0fa40d782e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:24.875951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddc994-8149-40ef-b9ca-0fa40d782e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:24.877292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddc994-8149-40ef-b9ca-0fa40d782e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:24.877292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddc994-8149-40ef-b9ca-0fa40d782e92 admin admin] PATH_INFO: `/v3/users/13aa318a9a37496385e7df324c6a1c50` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:24.878413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddc994-8149-40ef-b9ca-0fa40d782e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13aa318a9a37496385e7df324c6a1c50)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:24.880376 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddc994-8149-40ef-b9ca-0fa40d782e92 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:24.948938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:25.037013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7fddc994-8149-40ef-b9ca-0fa40d782e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13aa318a9a37496385e7df324c6a1c50'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:25.049124 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7fddc994-8149-40ef-b9ca-0fa40d782e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13aa318a9a37496385e7df324c6a1c50'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:25.164553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7fddc994-8149-40ef-b9ca-0fa40d782e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13aa318a9a37496385e7df324c6a1c50'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:25.165565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7fddc994-8149-40ef-b9ca-0fa40d782e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13aa318a9a37496385e7df324c6a1c50'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:25.177333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7fddc994-8149-40ef-b9ca-0fa40d782e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13aa318a9a37496385e7df324c6a1c50'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:25.183593 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1258/2486] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:48:24 2022] DELETE /identity/v3/users/13aa318a9a37496385e7df324c6a1c50 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:25.192494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a73b5d-2f10-4ebb-872a-a90d50455b49 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:25.192494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a73b5d-2f10-4ebb-872a-a90d50455b49 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:25.192494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a73b5d-2f10-4ebb-872a-a90d50455b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:25.207826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-13a73b5d-2f10-4ebb-872a-a90d50455b49 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:25.209127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-13a73b5d-2f10-4ebb-872a-a90d50455b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:25.266255 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1229/2487] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:48:25.276435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddc994-8149-40ef-b9ca-0fa40d782e92 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:25.277928 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7fddc994-8149-40ef-b9ca-0fa40d782e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:25.336064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddc994-8149-40ef-b9ca-0fa40d782e92 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:25.339676 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87a5d6-b950-4012-bdf5-29e347decba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:25.340325 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87a5d6-b950-4012-bdf5-29e347decba6 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:25.340615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87a5d6-b950-4012-bdf5-29e347decba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:25.340898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87a5d6-b950-4012-bdf5-29e347decba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:25.350113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87a5d6-b950-4012-bdf5-29e347decba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:25.351535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87a5d6-b950-4012-bdf5-29e347decba6 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:25.361027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1259/2488] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:25.812495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a73b5d-2f10-4ebb-872a-a90d50455b49 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:25.814692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-13a73b5d-2f10-4ebb-872a-a90d50455b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:25.826618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a73b5d-2f10-4ebb-872a-a90d50455b49 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:25.831326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940a692-4331-414f-8a50-9a1e56a205cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:25.832193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940a692-4331-414f-8a50-9a1e56a205cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:25.832473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940a692-4331-414f-8a50-9a1e56a205cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:25.832752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940a692-4331-414f-8a50-9a1e56a205cb admin admin] PATH_INFO: `/v3/projects/060da834e0574299a5712508e54d7ba7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:25.833963 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940a692-4331-414f-8a50-9a1e56a205cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=060da834e0574299a5712508e54d7ba7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:25.835957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940a692-4331-414f-8a50-9a1e56a205cb admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:25.972113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3940a692-4331-414f-8a50-9a1e56a205cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '060da834e0574299a5712508e54d7ba7', 'request_id': 'req-3940a692-4331-414f-8a50-9a1e56a205cb'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:25.977487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3940a692-4331-414f-8a50-9a1e56a205cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '060da834e0574299a5712508e54d7ba7', 'request_id': 'req-3940a692-4331-414f-8a50-9a1e56a205cb'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:25.983582 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3940a692-4331-414f-8a50-9a1e56a205cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '060da834e0574299a5712508e54d7ba7', 'request_id': 'req-3940a692-4331-414f-8a50-9a1e56a205cb'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:26.061805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3940a692-4331-414f-8a50-9a1e56a205cb admin admin] The token cache is being invalidate because project 060da834e0574299a5712508e54d7ba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:48:26.062105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3940a692-4331-414f-8a50-9a1e56a205cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:26.062738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3940a692-4331-414f-8a50-9a1e56a205cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:26.064625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1230/2489] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:48:25 2022] DELETE /identity/v3/projects/060da834e0574299a5712508e54d7ba7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:26.068556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87a5d6-b950-4012-bdf5-29e347decba6 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:26.078589 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87a5d6-b950-4012-bdf5-29e347decba6 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:26.138895 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bda56-9624-457c-907b-62da73329077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:26.139780 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bda56-9624-457c-907b-62da73329077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:26.140066 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bda56-9624-457c-907b-62da73329077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:26.140348 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bda56-9624-457c-907b-62da73329077 admin admin] PATH_INFO: `/v3/users/1b5be7e638b34090845fc9ab13aa5c06` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:26.141583 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bda56-9624-457c-907b-62da73329077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b5be7e638b34090845fc9ab13aa5c06)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:26.143787 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bda56-9624-457c-907b-62da73329077 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:26.396557 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7d3bda56-9624-457c-907b-62da73329077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b5be7e638b34090845fc9ab13aa5c06'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:26.397616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:26.409390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7d3bda56-9624-457c-907b-62da73329077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b5be7e638b34090845fc9ab13aa5c06'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:26.496228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7d3bda56-9624-457c-907b-62da73329077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b5be7e638b34090845fc9ab13aa5c06'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:26.496842 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7d3bda56-9624-457c-907b-62da73329077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b5be7e638b34090845fc9ab13aa5c06'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:26.504288 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7d3bda56-9624-457c-907b-62da73329077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b5be7e638b34090845fc9ab13aa5c06'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:26.510163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1260/2490] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:48:26 2022] DELETE /identity/v3/users/1b5be7e638b34090845fc9ab13aa5c06 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:26.984080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940a692-4331-414f-8a50-9a1e56a205cb admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:26.988951 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3940a692-4331-414f-8a50-9a1e56a205cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:26.999347 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940a692-4331-414f-8a50-9a1e56a205cb admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:27.028544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:27.070339 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f5ccb-8d49-4f08-935c-cbfc1c19d506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:27.071214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f5ccb-8d49-4f08-935c-cbfc1c19d506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:27.071508 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f5ccb-8d49-4f08-935c-cbfc1c19d506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:27.071793 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f5ccb-8d49-4f08-935c-cbfc1c19d506 admin admin] PATH_INFO: `/v3/projects/102f0d9088174e77bb51473facfedc6c` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:27.073551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f5ccb-8d49-4f08-935c-cbfc1c19d506 admin admin] RBAC: Authorizing `identity:delete_project(project_id=102f0d9088174e77bb51473facfedc6c)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:27.075478 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f5ccb-8d49-4f08-935c-cbfc1c19d506 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:27.178486 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3e5f5ccb-8d49-4f08-935c-cbfc1c19d506 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '102f0d9088174e77bb51473facfedc6c', 'request_id': 'req-3e5f5ccb-8d49-4f08-935c-cbfc1c19d506'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:27.187223 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3e5f5ccb-8d49-4f08-935c-cbfc1c19d506 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '102f0d9088174e77bb51473facfedc6c', 'request_id': 'req-3e5f5ccb-8d49-4f08-935c-cbfc1c19d506'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:27.192469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3e5f5ccb-8d49-4f08-935c-cbfc1c19d506 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '102f0d9088174e77bb51473facfedc6c', 'request_id': 'req-3e5f5ccb-8d49-4f08-935c-cbfc1c19d506'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:27.317197 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3e5f5ccb-8d49-4f08-935c-cbfc1c19d506 admin admin] The token cache is being invalidate because project 102f0d9088174e77bb51473facfedc6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:48:27.317670 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3e5f5ccb-8d49-4f08-935c-cbfc1c19d506 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:27.318790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3e5f5ccb-8d49-4f08-935c-cbfc1c19d506 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:27.320663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1231/2491] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:48:26 2022] DELETE /identity/v3/projects/102f0d9088174e77bb51473facfedc6c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:27.325597 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd0731-71ea-47d8-8b6a-bc915ae311fb None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:27.326125 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd0731-71ea-47d8-8b6a-bc915ae311fb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:27.326239 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd0731-71ea-47d8-8b6a-bc915ae311fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:27.345520 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-bcfd0731-71ea-47d8-8b6a-bc915ae311fb None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:27.346759 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bcfd0731-71ea-47d8-8b6a-bc915ae311fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:27.417698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1261/2492] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:27.426535 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce59d4-d883-453d-83c5-f603fb76ea4a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:27.426826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce59d4-d883-453d-83c5-f603fb76ea4a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:27.427107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce59d4-d883-453d-83c5-f603fb76ea4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:27.446948 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-15ce59d4-d883-453d-83c5-f603fb76ea4a None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:27.448090 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-15ce59d4-d883-453d-83c5-f603fb76ea4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:27.456647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1232/2493] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:48:27.460933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd0731-71ea-47d8-8b6a-bc915ae311fb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:27.471775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd0731-71ea-47d8-8b6a-bc915ae311fb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:27.476664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80ed78-4b31-4b38-8850-fb0a7a34d12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:27.477387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80ed78-4b31-4b38-8850-fb0a7a34d12c admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:27.477679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80ed78-4b31-4b38-8850-fb0a7a34d12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:27.478121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80ed78-4b31-4b38-8850-fb0a7a34d12c admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:27.478866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80ed78-4b31-4b38-8850-fb0a7a34d12c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:48:27.479211 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80ed78-4b31-4b38-8850-fb0a7a34d12c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:27.480821 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80ed78-4b31-4b38-8850-fb0a7a34d12c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:27.497513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1262/2494] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:48:27.498708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce59d4-d883-453d-83c5-f603fb76ea4a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:27.508699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce59d4-d883-453d-83c5-f603fb76ea4a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:27.513675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df96b7-bbc8-4924-b434-81b0a4988312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:27.514751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df96b7-bbc8-4924-b434-81b0a4988312 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:27.514751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df96b7-bbc8-4924-b434-81b0a4988312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:27.514966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df96b7-bbc8-4924-b434-81b0a4988312 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:27.515566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df96b7-bbc8-4924-b434-81b0a4988312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:27.518117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df96b7-bbc8-4924-b434-81b0a4988312 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:27.619213 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1233/2495] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:48:27 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:48:27.623462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80ed78-4b31-4b38-8850-fb0a7a34d12c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:27.634385 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80ed78-4b31-4b38-8850-fb0a7a34d12c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:27.694357 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343bd9a-0b02-4df1-9ae0-4a900760714c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:27.695049 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343bd9a-0b02-4df1-9ae0-4a900760714c admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:27.695363 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343bd9a-0b02-4df1-9ae0-4a900760714c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:27.695645 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343bd9a-0b02-4df1-9ae0-4a900760714c admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:27.696371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343bd9a-0b02-4df1-9ae0-4a900760714c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:27.698430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343bd9a-0b02-4df1-9ae0-4a900760714c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:27.777004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1263/2496] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:48:27 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:48:27.780957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df96b7-bbc8-4924-b434-81b0a4988312 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:27.793735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df96b7-bbc8-4924-b434-81b0a4988312 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:27.798737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd18c6-0e76-405d-99b2-a2b906973e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:27.799664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd18c6-0e76-405d-99b2-a2b906973e3e admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:27.799950 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd18c6-0e76-405d-99b2-a2b906973e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:27.800225 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd18c6-0e76-405d-99b2-a2b906973e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:27.800949 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd18c6-0e76-405d-99b2-a2b906973e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:27.802624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd18c6-0e76-405d-99b2-a2b906973e3e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:27.813516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1234/2497] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:48:27.817446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343bd9a-0b02-4df1-9ae0-4a900760714c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:27.829850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343bd9a-0b02-4df1-9ae0-4a900760714c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:27.834742 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa7816-3b24-48dc-b355-8a392e1b0198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:27.835421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa7816-3b24-48dc-b355-8a392e1b0198 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:27.835739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa7816-3b24-48dc-b355-8a392e1b0198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:27.836022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa7816-3b24-48dc-b355-8a392e1b0198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:27.836848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa7816-3b24-48dc-b355-8a392e1b0198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:27.838588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa7816-3b24-48dc-b355-8a392e1b0198 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:27.849832 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1264/2498] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:48:27.853782 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd18c6-0e76-405d-99b2-a2b906973e3e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:27.864568 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd18c6-0e76-405d-99b2-a2b906973e3e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:27.869326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ac292-9100-454b-a253-f609fa643d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:27.870212 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ac292-9100-454b-a253-f609fa643d67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:27.870620 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ac292-9100-454b-a253-f609fa643d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:27.870906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ac292-9100-454b-a253-f609fa643d67 admin admin] PATH_INFO: `/v3/projects/cd9250eba0e14d0a907934678e4cb2c6/users/ccb96ff5f02a4a0ba510b705fa948dca/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:27.885419 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ac292-9100-454b-a253-f609fa643d67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd9250eba0e14d0a907934678e4cb2c6, user_id=ccb96ff5f02a4a0ba510b705fa948dca, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:27.888094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ac292-9100-454b-a253-f609fa643d67 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:27.994179 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1235/2499] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:48:27 2022] PUT /identity/v3/projects/cd9250eba0e14d0a907934678e4cb2c6/users/ccb96ff5f02a4a0ba510b705fa948dca/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:48:28.005733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c2614-fa9e-4259-8662-378bf18c5474 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:28.006117 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c2614-fa9e-4259-8662-378bf18c5474 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:28.006476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c2614-fa9e-4259-8662-378bf18c5474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:28.035111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-011c2614-fa9e-4259-8662-378bf18c5474 None None] MFA Rules not processed for user `ccb96ff5f02a4a0ba510b705fa948dca`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:28.036326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-011c2614-fa9e-4259-8662-378bf18c5474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:28.056136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:28.102790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1265/2500] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:28.107464 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ac292-9100-454b-a253-f609fa643d67 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:28.158222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:28.175434 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ac292-9100-454b-a253-f609fa643d67 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:28.184270 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36a5e9-8a75-43c8-b0bb-4fb2e0db8b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:28.185178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36a5e9-8a75-43c8-b0bb-4fb2e0db8b60 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:28.185428 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36a5e9-8a75-43c8-b0bb-4fb2e0db8b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:28.185704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36a5e9-8a75-43c8-b0bb-4fb2e0db8b60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:28.186454 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b36a5e9-8a75-43c8-b0bb-4fb2e0db8b60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:28.188999 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b36a5e9-8a75-43c8-b0bb-4fb2e0db8b60 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:28.381029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1236/2501] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:48:28 2022] POST /identity/v3/projects => generated 387 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:48:28.382242 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c2614-fa9e-4259-8662-378bf18c5474 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:28.396390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c2614-fa9e-4259-8662-378bf18c5474 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:28.455981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8d4ef-0c6d-4c5c-9c00-19c3bf13c8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:28.456807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8d4ef-0c6d-4c5c-9c00-19c3bf13c8a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:28.457196 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8d4ef-0c6d-4c5c-9c00-19c3bf13c8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:28.457480 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8d4ef-0c6d-4c5c-9c00-19c3bf13c8a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:28.458180 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8d4ef-0c6d-4c5c-9c00-19c3bf13c8a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:28.460145 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8d4ef-0c6d-4c5c-9c00-19c3bf13c8a9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:28.560000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1266/2502] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:48:28 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:48:28.568497 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36a5e9-8a75-43c8-b0bb-4fb2e0db8b60 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:28.578800 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36a5e9-8a75-43c8-b0bb-4fb2e0db8b60 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:28.583818 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7c5c0-0420-4372-ad20-a0af56c7f7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:28.584438 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7c5c0-0420-4372-ad20-a0af56c7f7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:28.584893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7c5c0-0420-4372-ad20-a0af56c7f7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:28.585016 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7c5c0-0420-4372-ad20-a0af56c7f7b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:28.585770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7c5c0-0420-4372-ad20-a0af56c7f7b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:28.587384 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7c5c0-0420-4372-ad20-a0af56c7f7b3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:28.598477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1237/2503] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:48:28.603720 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8d4ef-0c6d-4c5c-9c00-19c3bf13c8a9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:28.618022 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8d4ef-0c6d-4c5c-9c00-19c3bf13c8a9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:28.623158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21fe40-148f-4a56-9ad7-48490d4ff2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:28.624463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21fe40-148f-4a56-9ad7-48490d4ff2c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:28.624749 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21fe40-148f-4a56-9ad7-48490d4ff2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:28.625032 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21fe40-148f-4a56-9ad7-48490d4ff2c6 admin admin] PATH_INFO: `/v3/projects/c7358c160e9a431e924d0cbe6b3470fe/users/8d69d964c13e4687b76cfbdb40bc8ec2/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:28.641558 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21fe40-148f-4a56-9ad7-48490d4ff2c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7358c160e9a431e924d0cbe6b3470fe, user_id=8d69d964c13e4687b76cfbdb40bc8ec2, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:28.644863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21fe40-148f-4a56-9ad7-48490d4ff2c6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:28.760048 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1267/2504] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:48:28 2022] PUT /identity/v3/projects/c7358c160e9a431e924d0cbe6b3470fe/users/8d69d964c13e4687b76cfbdb40bc8ec2/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:48:28.763983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7c5c0-0420-4372-ad20-a0af56c7f7b3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:28.774754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7c5c0-0420-4372-ad20-a0af56c7f7b3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:28.833645 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d93e4d-5344-4d23-a05d-47698f0eb14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:28.834241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d93e4d-5344-4d23-a05d-47698f0eb14e admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:28.834526 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d93e4d-5344-4d23-a05d-47698f0eb14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:28.834831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d93e4d-5344-4d23-a05d-47698f0eb14e admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:28.835575 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d93e4d-5344-4d23-a05d-47698f0eb14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:28.837495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d93e4d-5344-4d23-a05d-47698f0eb14e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:28.847663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1238/2505] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:48:28.851793 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21fe40-148f-4a56-9ad7-48490d4ff2c6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:28.863096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21fe40-148f-4a56-9ad7-48490d4ff2c6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:28.868054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f999c-4172-4ba1-93b4-3895b2ac3b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:28.868950 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f999c-4172-4ba1-93b4-3895b2ac3b5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:28.869275 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f999c-4172-4ba1-93b4-3895b2ac3b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:28.869563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f999c-4172-4ba1-93b4-3895b2ac3b5f admin admin] PATH_INFO: `/v3/projects/c7358c160e9a431e924d0cbe6b3470fe/users/8d69d964c13e4687b76cfbdb40bc8ec2/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:28.871699 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f999c-4172-4ba1-93b4-3895b2ac3b5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7358c160e9a431e924d0cbe6b3470fe, user_id=8d69d964c13e4687b76cfbdb40bc8ec2, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:28.874575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f999c-4172-4ba1-93b4-3895b2ac3b5f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:28.886020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1268/2506] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:48:28 2022] PUT /identity/v3/projects/c7358c160e9a431e924d0cbe6b3470fe/users/8d69d964c13e4687b76cfbdb40bc8ec2/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:28.897878 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ace668-ab2d-4308-b9a3-8d405d171474 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:28.898739 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ace668-ab2d-4308-b9a3-8d405d171474 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:28.899093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ace668-ab2d-4308-b9a3-8d405d171474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:28.929207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a0ace668-ab2d-4308-b9a3-8d405d171474 None None] MFA Rules not processed for user `8d69d964c13e4687b76cfbdb40bc8ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:28.930416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a0ace668-ab2d-4308-b9a3-8d405d171474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:28.999136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1239/2507] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:29.009875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f999c-4172-4ba1-93b4-3895b2ac3b5f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:29.011562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a99f999c-4172-4ba1-93b4-3895b2ac3b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:29.073898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f999c-4172-4ba1-93b4-3895b2ac3b5f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:29.077693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79216fe-137b-47ca-a8a1-c8f6a400ba77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:29.078453 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79216fe-137b-47ca-a8a1-c8f6a400ba77 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:29.078748 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79216fe-137b-47ca-a8a1-c8f6a400ba77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:29.079030 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79216fe-137b-47ca-a8a1-c8f6a400ba77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:29.089461 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79216fe-137b-47ca-a8a1-c8f6a400ba77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:29.090827 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79216fe-137b-47ca-a8a1-c8f6a400ba77 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:29.101587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1269/2508] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:30.368219 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ace668-ab2d-4308-b9a3-8d405d171474 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:30.418265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:30.437012 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ace668-ab2d-4308-b9a3-8d405d171474 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:30.442217 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a10a6-1659-4c6c-8d5d-b5f84713813d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:30.443090 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a10a6-1659-4c6c-8d5d-b5f84713813d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:30.443447 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a10a6-1659-4c6c-8d5d-b5f84713813d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:30.443778 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a10a6-1659-4c6c-8d5d-b5f84713813d admin admin] PATH_INFO: `/v3/users/ccb96ff5f02a4a0ba510b705fa948dca` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:30.445000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a10a6-1659-4c6c-8d5d-b5f84713813d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccb96ff5f02a4a0ba510b705fa948dca)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:30.449489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a10a6-1659-4c6c-8d5d-b5f84713813d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:30.658860 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4c5a10a6-1659-4c6c-8d5d-b5f84713813d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccb96ff5f02a4a0ba510b705fa948dca'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:30.659510 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4c5a10a6-1659-4c6c-8d5d-b5f84713813d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccb96ff5f02a4a0ba510b705fa948dca'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:30.660112 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4c5a10a6-1659-4c6c-8d5d-b5f84713813d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccb96ff5f02a4a0ba510b705fa948dca'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:30.666935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4c5a10a6-1659-4c6c-8d5d-b5f84713813d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccb96ff5f02a4a0ba510b705fa948dca'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:30.679586 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-4c5a10a6-1659-4c6c-8d5d-b5f84713813d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccb96ff5f02a4a0ba510b705fa948dca'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:30.764962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1240/2509] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:48:30 2022] DELETE /identity/v3/users/ccb96ff5f02a4a0ba510b705fa948dca => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:48:30.770225 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b560a64-a9c4-40da-84dd-f4bbe0f13b23 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:30.770705 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b560a64-a9c4-40da-84dd-f4bbe0f13b23 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:30.771032 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b560a64-a9c4-40da-84dd-f4bbe0f13b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:30.789607 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5b560a64-a9c4-40da-84dd-f4bbe0f13b23 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:30.790847 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5b560a64-a9c4-40da-84dd-f4bbe0f13b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:30.800370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:30.851068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1270/2510] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:30.860909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a10a6-1659-4c6c-8d5d-b5f84713813d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:30.862503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4c5a10a6-1659-4c6c-8d5d-b5f84713813d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:30.924619 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a10a6-1659-4c6c-8d5d-b5f84713813d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:30.934728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56fba4-f30d-4cd8-9878-85000dde1b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:30.935314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56fba4-f30d-4cd8-9878-85000dde1b80 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:30.935604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56fba4-f30d-4cd8-9878-85000dde1b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:30.935884 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56fba4-f30d-4cd8-9878-85000dde1b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:30.945886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56fba4-f30d-4cd8-9878-85000dde1b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:30.947316 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56fba4-f30d-4cd8-9878-85000dde1b80 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:30.958020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1241/2511] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:31.395540 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b560a64-a9c4-40da-84dd-f4bbe0f13b23 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:31.396985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5b560a64-a9c4-40da-84dd-f4bbe0f13b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:31.410422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b560a64-a9c4-40da-84dd-f4bbe0f13b23 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:31.415223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:31.416100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:31.416398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:31.416677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421 admin admin] PATH_INFO: `/v3/projects/cd9250eba0e14d0a907934678e4cb2c6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:31.417977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd9250eba0e14d0a907934678e4cb2c6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:31.419867 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:31.587708 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd9250eba0e14d0a907934678e4cb2c6', 'request_id': 'req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:31.594138 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd9250eba0e14d0a907934678e4cb2c6', 'request_id': 'req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:31.644383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd9250eba0e14d0a907934678e4cb2c6', 'request_id': 'req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:31.654961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421 admin admin] The token cache is being invalidate because project cd9250eba0e14d0a907934678e4cb2c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:48:31.655255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:31.655884 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:31.657718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1271/2512] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:48:31 2022] DELETE /identity/v3/projects/cd9250eba0e14d0a907934678e4cb2c6 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:48:31.661593 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56fba4-f30d-4cd8-9878-85000dde1b80 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:31.671323 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56fba4-f30d-4cd8-9878-85000dde1b80 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:31.728917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe9f57-f4e6-4a7d-b057-826fad618c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:31.729817 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe9f57-f4e6-4a7d-b057-826fad618c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:31.730102 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe9f57-f4e6-4a7d-b057-826fad618c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:31.730383 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe9f57-f4e6-4a7d-b057-826fad618c1a admin admin] PATH_INFO: `/v3/users/8d69d964c13e4687b76cfbdb40bc8ec2` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:31.731555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe9f57-f4e6-4a7d-b057-826fad618c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d69d964c13e4687b76cfbdb40bc8ec2)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:31.733506 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe9f57-f4e6-4a7d-b057-826fad618c1a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:31.740381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:31.955958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2dfe9f57-f4e6-4a7d-b057-826fad618c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d69d964c13e4687b76cfbdb40bc8ec2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:31.956563 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2dfe9f57-f4e6-4a7d-b057-826fad618c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d69d964c13e4687b76cfbdb40bc8ec2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:31.957196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2dfe9f57-f4e6-4a7d-b057-826fad618c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d69d964c13e4687b76cfbdb40bc8ec2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:31.963811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2dfe9f57-f4e6-4a7d-b057-826fad618c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d69d964c13e4687b76cfbdb40bc8ec2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:31.976985 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2dfe9f57-f4e6-4a7d-b057-826fad618c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d69d964c13e4687b76cfbdb40bc8ec2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:32.068428 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1242/2513] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:48:31 2022] DELETE /identity/v3/users/8d69d964c13e4687b76cfbdb40bc8ec2 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:32.801658 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:32.803138 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:32.816893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec2681-5d60-4ae0-b27f-1a2e1a4fb421 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:32.874032 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2c12b-48b4-4039-be3e-9c1332e29bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:32.874891 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2c12b-48b4-4039-be3e-9c1332e29bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:32.875173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2c12b-48b4-4039-be3e-9c1332e29bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:32.875447 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2c12b-48b4-4039-be3e-9c1332e29bd6 admin admin] PATH_INFO: `/v3/projects/c7358c160e9a431e924d0cbe6b3470fe` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:32.877454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2c12b-48b4-4039-be3e-9c1332e29bd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7358c160e9a431e924d0cbe6b3470fe)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:32.879392 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2c12b-48b4-4039-be3e-9c1332e29bd6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:32.999543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:33.006064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cec2c12b-48b4-4039-be3e-9c1332e29bd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7358c160e9a431e924d0cbe6b3470fe', 'request_id': 'req-cec2c12b-48b4-4039-be3e-9c1332e29bd6'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:33.012140 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cec2c12b-48b4-4039-be3e-9c1332e29bd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7358c160e9a431e924d0cbe6b3470fe', 'request_id': 'req-cec2c12b-48b4-4039-be3e-9c1332e29bd6'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:33.165444 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cec2c12b-48b4-4039-be3e-9c1332e29bd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7358c160e9a431e924d0cbe6b3470fe', 'request_id': 'req-cec2c12b-48b4-4039-be3e-9c1332e29bd6'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:33.177772 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cec2c12b-48b4-4039-be3e-9c1332e29bd6 admin admin] The token cache is being invalidate because project c7358c160e9a431e924d0cbe6b3470fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:48:33.189850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cec2c12b-48b4-4039-be3e-9c1332e29bd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:33.189850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-cec2c12b-48b4-4039-be3e-9c1332e29bd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:33.189850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1272/2514] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:48:32 2022] DELETE /identity/v3/projects/c7358c160e9a431e924d0cbe6b3470fe => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:33.190183 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a58ec6-6713-4317-a8a1-115d01047a0a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:33.190183 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a58ec6-6713-4317-a8a1-115d01047a0a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:33.190183 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a58ec6-6713-4317-a8a1-115d01047a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:33.205970 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c8a58ec6-6713-4317-a8a1-115d01047a0a None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:33.207172 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c8a58ec6-6713-4317-a8a1-115d01047a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:33.269401 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1243/2515] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:33.278087 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16be62e-4f36-4a97-8055-5cd8e90a0418 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:33.278596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16be62e-4f36-4a97-8055-5cd8e90a0418 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:33.278959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16be62e-4f36-4a97-8055-5cd8e90a0418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:33.298574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c16be62e-4f36-4a97-8055-5cd8e90a0418 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:33.300051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c16be62e-4f36-4a97-8055-5cd8e90a0418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:33.308374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1273/2516] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:33.312592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a58ec6-6713-4317-a8a1-115d01047a0a None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:33.326057 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a58ec6-6713-4317-a8a1-115d01047a0a None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:33.330822 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee19c06-4d0d-4986-ac9a-97fbace5ff81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:33.331479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee19c06-4d0d-4986-ac9a-97fbace5ff81 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:33.331776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee19c06-4d0d-4986-ac9a-97fbace5ff81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:33.332058 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee19c06-4d0d-4986-ac9a-97fbace5ff81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:33.332790 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee19c06-4d0d-4986-ac9a-97fbace5ff81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:48:33.333138 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee19c06-4d0d-4986-ac9a-97fbace5ff81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:33.335164 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee19c06-4d0d-4986-ac9a-97fbace5ff81 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:33.349083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1244/2517] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:48:33.353430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16be62e-4f36-4a97-8055-5cd8e90a0418 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:33.363777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16be62e-4f36-4a97-8055-5cd8e90a0418 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:33.368365 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0656aeb-bf09-4ed5-8ebb-4990946a117b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:33.369020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0656aeb-bf09-4ed5-8ebb-4990946a117b admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:33.369347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0656aeb-bf09-4ed5-8ebb-4990946a117b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:33.369629 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0656aeb-bf09-4ed5-8ebb-4990946a117b admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:33.370318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0656aeb-bf09-4ed5-8ebb-4990946a117b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:33.372219 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0656aeb-bf09-4ed5-8ebb-4990946a117b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:33.503647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1274/2518] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:48:33 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:48:33.507727 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee19c06-4d0d-4986-ac9a-97fbace5ff81 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:33.517990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee19c06-4d0d-4986-ac9a-97fbace5ff81 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:33.574992 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631eacd5-72d7-4bad-9f87-a52ba5ddab41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:33.575662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631eacd5-72d7-4bad-9f87-a52ba5ddab41 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:33.575948 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631eacd5-72d7-4bad-9f87-a52ba5ddab41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:33.576229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631eacd5-72d7-4bad-9f87-a52ba5ddab41 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:33.576918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631eacd5-72d7-4bad-9f87-a52ba5ddab41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:33.578919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631eacd5-72d7-4bad-9f87-a52ba5ddab41 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:33.667925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1245/2519] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:48:33 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:48:33.672150 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0656aeb-bf09-4ed5-8ebb-4990946a117b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:33.683878 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0656aeb-bf09-4ed5-8ebb-4990946a117b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:33.688980 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639629f-bd7d-4f7a-aded-1b7bef6d65fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:33.689584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639629f-bd7d-4f7a-aded-1b7bef6d65fd admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:33.689879 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639629f-bd7d-4f7a-aded-1b7bef6d65fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:33.690192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639629f-bd7d-4f7a-aded-1b7bef6d65fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:33.690955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639629f-bd7d-4f7a-aded-1b7bef6d65fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:33.692584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639629f-bd7d-4f7a-aded-1b7bef6d65fd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:33.704240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1275/2520] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:48:33.710407 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631eacd5-72d7-4bad-9f87-a52ba5ddab41 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:33.715645 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:33.721447 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631eacd5-72d7-4bad-9f87-a52ba5ddab41 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:33.726175 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177157f-50d8-4ac1-9b3f-c0c6f7a860be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:33.726839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177157f-50d8-4ac1-9b3f-c0c6f7a860be admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:33.727126 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177157f-50d8-4ac1-9b3f-c0c6f7a860be admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:33.727410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177157f-50d8-4ac1-9b3f-c0c6f7a860be admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:33.728159 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177157f-50d8-4ac1-9b3f-c0c6f7a860be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:33.729898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177157f-50d8-4ac1-9b3f-c0c6f7a860be admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:33.741818 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1246/2521] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:48:33.749601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639629f-bd7d-4f7a-aded-1b7bef6d65fd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:33.760163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639629f-bd7d-4f7a-aded-1b7bef6d65fd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:33.764802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d972b9-5859-4b0d-829e-f93b58678975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:33.765714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d972b9-5859-4b0d-829e-f93b58678975 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:33.766004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d972b9-5859-4b0d-829e-f93b58678975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:33.766285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d972b9-5859-4b0d-829e-f93b58678975 admin admin] PATH_INFO: `/v3/projects/780ecbc29b1b455cadc088113b086366/users/3a87753b939547aba2fa831333580e43/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:33.783009 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d972b9-5859-4b0d-829e-f93b58678975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=780ecbc29b1b455cadc088113b086366, user_id=3a87753b939547aba2fa831333580e43, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:33.785746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d972b9-5859-4b0d-829e-f93b58678975 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:33.925618 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1276/2522] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:48:33 2022] PUT /identity/v3/projects/780ecbc29b1b455cadc088113b086366/users/3a87753b939547aba2fa831333580e43/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:33.930636 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bce45-c660-483c-94ad-75a07058e2eb None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:33.930931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bce45-c660-483c-94ad-75a07058e2eb None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:33.931218 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bce45-c660-483c-94ad-75a07058e2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:33.955517 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ab4bce45-c660-483c-94ad-75a07058e2eb None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:33.955517 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ab4bce45-c660-483c-94ad-75a07058e2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:34.018791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1247/2523] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:34.028218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d972b9-5859-4b0d-829e-f93b58678975 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:34.030027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-50d972b9-5859-4b0d-829e-f93b58678975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:34.088186 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d972b9-5859-4b0d-829e-f93b58678975 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:34.091883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73217dd-7748-4e85-9f62-80c0bb6bb1da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:34.092530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73217dd-7748-4e85-9f62-80c0bb6bb1da service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:34.092824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73217dd-7748-4e85-9f62-80c0bb6bb1da service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:34.093147 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73217dd-7748-4e85-9f62-80c0bb6bb1da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:34.102957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73217dd-7748-4e85-9f62-80c0bb6bb1da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:34.104496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73217dd-7748-4e85-9f62-80c0bb6bb1da service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:34.114177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1277/2524] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:34.841521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fdce0-499a-4cdd-9684-b94311ed6658 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:34.842011 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fdce0-499a-4cdd-9684-b94311ed6658 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:34.842125 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fdce0-499a-4cdd-9684-b94311ed6658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:34.872106 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c17fdce0-499a-4cdd-9684-b94311ed6658 None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:34.872106 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c17fdce0-499a-4cdd-9684-b94311ed6658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:34.917667 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1248/2525] 10.222.0.64 () {58 vars in 1009 bytes} [Mon May 23 15:48:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:35.078772 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73217dd-7748-4e85-9f62-80c0bb6bb1da service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:35.079958 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d73217dd-7748-4e85-9f62-80c0bb6bb1da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:35.149430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73217dd-7748-4e85-9f62-80c0bb6bb1da service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:35.154740 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07460a35-0ab2-4ab5-a1d0-58c33750dce5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:35.155253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07460a35-0ab2-4ab5-a1d0-58c33750dce5 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:35.155543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07460a35-0ab2-4ab5-a1d0-58c33750dce5 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:35.155825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07460a35-0ab2-4ab5-a1d0-58c33750dce5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:35.164525 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07460a35-0ab2-4ab5-a1d0-58c33750dce5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:35.166227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07460a35-0ab2-4ab5-a1d0-58c33750dce5 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:35.169766 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:35.175303 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1278/2526] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:48:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:48:35.565493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd056c-c471-443a-a5bc-ddd1b97cbad3 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:35.565793 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd056c-c471-443a-a5bc-ddd1b97cbad3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:35.566083 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd056c-c471-443a-a5bc-ddd1b97cbad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:35.613033 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-82fd056c-c471-443a-a5bc-ddd1b97cbad3 None None] MFA Rules not processed for user `3a87753b939547aba2fa831333580e43`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:35.614349 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-82fd056c-c471-443a-a5bc-ddd1b97cbad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:35.765336 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1249/2527] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:35.768601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07460a35-0ab2-4ab5-a1d0-58c33750dce5 service swift] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:35.788329 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07460a35-0ab2-4ab5-a1d0-58c33750dce5 service swift] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:35.794649 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb308848-4cab-484d-a4b3-24e375f52537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:35.795651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb308848-4cab-484d-a4b3-24e375f52537 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:35.795927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb308848-4cab-484d-a4b3-24e375f52537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:35.796230 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb308848-4cab-484d-a4b3-24e375f52537 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:35.797127 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb308848-4cab-484d-a4b3-24e375f52537 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:35.800654 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb308848-4cab-484d-a4b3-24e375f52537 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:36.100180 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1279/2528] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:48:35 2022] POST /identity/v3/projects => generated 369 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:48:36.104495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd056c-c471-443a-a5bc-ddd1b97cbad3 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:36.117821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd056c-c471-443a-a5bc-ddd1b97cbad3 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:36.146979 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=100 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:36.181173 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4a06f-9380-4ba5-9021-ea7a0dfd2140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:36.182248 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4a06f-9380-4ba5-9021-ea7a0dfd2140 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:36.182697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4a06f-9380-4ba5-9021-ea7a0dfd2140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:36.183320 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4a06f-9380-4ba5-9021-ea7a0dfd2140 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:36.184324 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4a06f-9380-4ba5-9021-ea7a0dfd2140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:36.187809 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4a06f-9380-4ba5-9021-ea7a0dfd2140 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:36.305108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1250/2529] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:48:36 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:48:36.308419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb308848-4cab-484d-a4b3-24e375f52537 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:36.322499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb308848-4cab-484d-a4b3-24e375f52537 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:36.327470 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46de71-7ecc-4522-8785-e0050042711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:36.328352 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46de71-7ecc-4522-8785-e0050042711e admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:36.328670 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46de71-7ecc-4522-8785-e0050042711e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:36.329001 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46de71-7ecc-4522-8785-e0050042711e admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:36.330042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46de71-7ecc-4522-8785-e0050042711e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:36.331728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46de71-7ecc-4522-8785-e0050042711e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:36.346081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1280/2530] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:48:36.350285 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4a06f-9380-4ba5-9021-ea7a0dfd2140 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:36.365392 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4a06f-9380-4ba5-9021-ea7a0dfd2140 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:36.369025 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373333cd-4750-4b36-9819-4b9e48e5d3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:36.370058 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373333cd-4750-4b36-9819-4b9e48e5d3d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:36.370542 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373333cd-4750-4b36-9819-4b9e48e5d3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:36.370898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373333cd-4750-4b36-9819-4b9e48e5d3d8 admin admin] PATH_INFO: `/v3/projects/9aaa48a2ec82422a8eb82cce558a778a/users/db01498679f74da4bf4beedf328f5da2/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:36.396596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373333cd-4750-4b36-9819-4b9e48e5d3d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aaa48a2ec82422a8eb82cce558a778a, user_id=db01498679f74da4bf4beedf328f5da2, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:36.396596 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373333cd-4750-4b36-9819-4b9e48e5d3d8 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:36.453188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1251/2531] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:48:36 2022] PUT /identity/v3/projects/9aaa48a2ec82422a8eb82cce558a778a/users/db01498679f74da4bf4beedf328f5da2/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:36.457021 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46de71-7ecc-4522-8785-e0050042711e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:36.468018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46de71-7ecc-4522-8785-e0050042711e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:36.536842 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6389e8-dcb8-4fbc-9a89-f5822cc1f381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:36.537563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6389e8-dcb8-4fbc-9a89-f5822cc1f381 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:36.537850 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6389e8-dcb8-4fbc-9a89-f5822cc1f381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:36.538134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6389e8-dcb8-4fbc-9a89-f5822cc1f381 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:36.538888 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6389e8-dcb8-4fbc-9a89-f5822cc1f381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:36.540535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6389e8-dcb8-4fbc-9a89-f5822cc1f381 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:36.554774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1281/2532] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:48:36.558961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373333cd-4750-4b36-9819-4b9e48e5d3d8 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:36.573234 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373333cd-4750-4b36-9819-4b9e48e5d3d8 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:36.578105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a188b-6f8e-4605-ba19-b89969c2a2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:36.579149 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a188b-6f8e-4605-ba19-b89969c2a2a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:36.579569 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a188b-6f8e-4605-ba19-b89969c2a2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:36.579861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a188b-6f8e-4605-ba19-b89969c2a2a9 admin admin] PATH_INFO: `/v3/projects/9aaa48a2ec82422a8eb82cce558a778a/users/db01498679f74da4bf4beedf328f5da2/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:36.582025 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a188b-6f8e-4605-ba19-b89969c2a2a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aaa48a2ec82422a8eb82cce558a778a, user_id=db01498679f74da4bf4beedf328f5da2, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:36.586538 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a188b-6f8e-4605-ba19-b89969c2a2a9 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:36.605536 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1252/2533] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:48:36 2022] PUT /identity/v3/projects/9aaa48a2ec82422a8eb82cce558a778a/users/db01498679f74da4bf4beedf328f5da2/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:38.311486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96078a9a-8f23-4459-8b18-8d91b0a183ea None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:38.312031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96078a9a-8f23-4459-8b18-8d91b0a183ea None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:38.312031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96078a9a-8f23-4459-8b18-8d91b0a183ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:38.355950 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-96078a9a-8f23-4459-8b18-8d91b0a183ea None None] MFA Rules not processed for user `db01498679f74da4bf4beedf328f5da2`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:38.357413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-96078a9a-8f23-4459-8b18-8d91b0a183ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:38.483559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1282/2534] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:38 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:38.496169 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a188b-6f8e-4605-ba19-b89969c2a2a9 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:38.497846 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-849a188b-6f8e-4605-ba19-b89969c2a2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:38.584776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a188b-6f8e-4605-ba19-b89969c2a2a9 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:38.588398 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776dfa8d-4cb9-43ef-92b2-9db14c6adbca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:38.589079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776dfa8d-4cb9-43ef-92b2-9db14c6adbca service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:38.589370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776dfa8d-4cb9-43ef-92b2-9db14c6adbca service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:38.589659 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776dfa8d-4cb9-43ef-92b2-9db14c6adbca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:38.659620 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776dfa8d-4cb9-43ef-92b2-9db14c6adbca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:38.661246 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776dfa8d-4cb9-43ef-92b2-9db14c6adbca service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:38.670412 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1253/2535] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:48:40.499722 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96078a9a-8f23-4459-8b18-8d91b0a183ea None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:40.501954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-96078a9a-8f23-4459-8b18-8d91b0a183ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:40.520906 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96078a9a-8f23-4459-8b18-8d91b0a183ea None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:40.526149 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34ecdf-8a2d-4687-aab4-11fc6f3b826e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:40.526824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34ecdf-8a2d-4687-aab4-11fc6f3b826e service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:40.527261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34ecdf-8a2d-4687-aab4-11fc6f3b826e service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:40.528284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34ecdf-8a2d-4687-aab4-11fc6f3b826e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:40.541346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34ecdf-8a2d-4687-aab4-11fc6f3b826e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:40.542090 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34ecdf-8a2d-4687-aab4-11fc6f3b826e service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:40.555434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1283/2536] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:40 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:44.125090 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776dfa8d-4cb9-43ef-92b2-9db14c6adbca service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:44.198724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:44.224690 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776dfa8d-4cb9-43ef-92b2-9db14c6adbca service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:44.228747 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b439b-ccb1-423d-80ba-7a0f58189914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:44.229643 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b439b-ccb1-423d-80ba-7a0f58189914 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:44.230292 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b439b-ccb1-423d-80ba-7a0f58189914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:44.230725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b439b-ccb1-423d-80ba-7a0f58189914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:44.233449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a99b439b-ccb1-423d-80ba-7a0f58189914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:44.297546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b439b-ccb1-423d-80ba-7a0f58189914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:44.308789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b439b-ccb1-423d-80ba-7a0f58189914 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:44.309505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1254/2537] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:50.551028 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=84,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:53.246069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34ecdf-8a2d-4687-aab4-11fc6f3b826e service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:53.247475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4d34ecdf-8a2d-4687-aab4-11fc6f3b826e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:53.260030 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34ecdf-8a2d-4687-aab4-11fc6f3b826e service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:53.263598 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30396ead-0574-4d00-bed2-fce82b375a9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:53.264230 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30396ead-0574-4d00-bed2-fce82b375a9f service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:53.264513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30396ead-0574-4d00-bed2-fce82b375a9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:53.264786 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30396ead-0574-4d00-bed2-fce82b375a9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:53.266493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-30396ead-0574-4d00-bed2-fce82b375a9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:53.332449 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30396ead-0574-4d00-bed2-fce82b375a9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:53.333928 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30396ead-0574-4d00-bed2-fce82b375a9f service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:53.343747 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1284/2538] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:54.198013 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd4079-fd67-4b1c-80ed-02a7a9c0939b None None] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:54.199680 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd4079-fd67-4b1c-80ed-02a7a9c0939b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:54.199680 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd4079-fd67-4b1c-80ed-02a7a9c0939b None None] PATH_INFO: `/` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:54.200207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1255/2539] 10.222.0.64 () {54 vars in 921 bytes} [Mon May 23 15:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:54.205768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5c455-11f9-407c-b7e5-4a80e73d8fe4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:54.207060 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5c455-11f9-407c-b7e5-4a80e73d8fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:54.207060 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5c455-11f9-407c-b7e5-4a80e73d8fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:54.227249 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-edc5c455-11f9-407c-b7e5-4a80e73d8fe4 None None] MFA Rules not processed for user `8da96bed73794fee8f719ec904936164`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:54.228783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-edc5c455-11f9-407c-b7e5-4a80e73d8fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:54.307885 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1285/2540] 10.222.0.64 () {58 vars in 1018 bytes} [Mon May 23 15:48:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:48:54.322437 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd4079-fd67-4b1c-80ed-02a7a9c0939b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:54.324288 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4ccd4079-fd67-4b1c-80ed-02a7a9c0939b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:54.393600 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd4079-fd67-4b1c-80ed-02a7a9c0939b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:54.397414 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90275605-fdb7-4f46-bf9a-89c0f75646db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee7a6c342548f98e7dc73f4f71a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:54.398065 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90275605-fdb7-4f46-bf9a-89c0f75646db service glance] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:54.398348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90275605-fdb7-4f46-bf9a-89c0f75646db service glance] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:54.398625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90275605-fdb7-4f46-bf9a-89c0f75646db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:54.410122 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90275605-fdb7-4f46-bf9a-89c0f75646db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:54.411908 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90275605-fdb7-4f46-bf9a-89c0f75646db service glance] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:54.425960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1256/2541] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:57.632080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5c455-11f9-407c-b7e5-4a80e73d8fe4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:57.633835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-edc5c455-11f9-407c-b7e5-4a80e73d8fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:57.705231 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5c455-11f9-407c-b7e5-4a80e73d8fe4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:57.710876 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a692028-55ea-4097-a69d-249a7dc14ece service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fbaeb7583a456d81cf79f6b101fb1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:57.711570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a692028-55ea-4097-a69d-249a7dc14ece service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:57.711862 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a692028-55ea-4097-a69d-249a7dc14ece service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:57.712163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a692028-55ea-4097-a69d-249a7dc14ece service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:57.714401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0a692028-55ea-4097-a69d-249a7dc14ece service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:57.726975 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a692028-55ea-4097-a69d-249a7dc14ece service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:57.728511 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a692028-55ea-4097-a69d-249a7dc14ece service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:57.739130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1286/2542] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:48:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:57.757641 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90275605-fdb7-4f46-bf9a-89c0f75646db service glance] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:57.774406 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90275605-fdb7-4f46-bf9a-89c0f75646db service glance] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:57.786245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91878d94-33d9-4f97-88c7-7697ead0a886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:57.788123 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91878d94-33d9-4f97-88c7-7697ead0a886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:57.788589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91878d94-33d9-4f97-88c7-7697ead0a886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:57.788977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91878d94-33d9-4f97-88c7-7697ead0a886 admin admin] PATH_INFO: `/v3/users/3a87753b939547aba2fa831333580e43` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:57.790725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91878d94-33d9-4f97-88c7-7697ead0a886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a87753b939547aba2fa831333580e43)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:57.793938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91878d94-33d9-4f97-88c7-7697ead0a886 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:58.007382 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-91878d94-33d9-4f97-88c7-7697ead0a886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a87753b939547aba2fa831333580e43'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:58.007957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-91878d94-33d9-4f97-88c7-7697ead0a886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a87753b939547aba2fa831333580e43'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:58.008638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-91878d94-33d9-4f97-88c7-7697ead0a886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a87753b939547aba2fa831333580e43'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:58.015501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-91878d94-33d9-4f97-88c7-7697ead0a886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a87753b939547aba2fa831333580e43'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:58.027909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-91878d94-33d9-4f97-88c7-7697ead0a886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a87753b939547aba2fa831333580e43'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:58.097396 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1257/2543] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:48:57 2022] DELETE /identity/v3/users/3a87753b939547aba2fa831333580e43 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:48:58.367114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a692028-55ea-4097-a69d-249a7dc14ece service placement] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:58.379498 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a692028-55ea-4097-a69d-249a7dc14ece service placement] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:58.392775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:58.451933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1b483-ec89-49c2-bfea-d15acf26b916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:58.452845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1b483-ec89-49c2-bfea-d15acf26b916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:58.453167 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1b483-ec89-49c2-bfea-d15acf26b916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:58.453451 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1b483-ec89-49c2-bfea-d15acf26b916 admin admin] PATH_INFO: `/v3/projects/780ecbc29b1b455cadc088113b086366` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:58.454677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1b483-ec89-49c2-bfea-d15acf26b916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=780ecbc29b1b455cadc088113b086366)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:58.457573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1b483-ec89-49c2-bfea-d15acf26b916 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:58.553338 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d1a1b483-ec89-49c2-bfea-d15acf26b916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '780ecbc29b1b455cadc088113b086366', 'request_id': 'req-d1a1b483-ec89-49c2-bfea-d15acf26b916'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:58.562759 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d1a1b483-ec89-49c2-bfea-d15acf26b916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '780ecbc29b1b455cadc088113b086366', 'request_id': 'req-d1a1b483-ec89-49c2-bfea-d15acf26b916'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:58.610570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d1a1b483-ec89-49c2-bfea-d15acf26b916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '780ecbc29b1b455cadc088113b086366', 'request_id': 'req-d1a1b483-ec89-49c2-bfea-d15acf26b916'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:58.622407 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d1a1b483-ec89-49c2-bfea-d15acf26b916 admin admin] The token cache is being invalidate because project 780ecbc29b1b455cadc088113b086366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:48:58.622703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d1a1b483-ec89-49c2-bfea-d15acf26b916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:58.623357 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d1a1b483-ec89-49c2-bfea-d15acf26b916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:58.624950 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1287/2544] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:48:58 2022] DELETE /identity/v3/projects/780ecbc29b1b455cadc088113b086366 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:58.629626 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91878d94-33d9-4f97-88c7-7697ead0a886 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:58.631056 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-91878d94-33d9-4f97-88c7-7697ead0a886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:58.639310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:58.644614 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91878d94-33d9-4f97-88c7-7697ead0a886 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:58.712558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372b725-27ef-4784-be0c-256de46f85aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:58.713661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372b725-27ef-4784-be0c-256de46f85aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:58.714385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372b725-27ef-4784-be0c-256de46f85aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:58.714673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372b725-27ef-4784-be0c-256de46f85aa admin admin] PATH_INFO: `/v3/users/db01498679f74da4bf4beedf328f5da2` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:58.715930 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372b725-27ef-4784-be0c-256de46f85aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=db01498679f74da4bf4beedf328f5da2)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:58.718057 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372b725-27ef-4784-be0c-256de46f85aa admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:58.916683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6372b725-27ef-4784-be0c-256de46f85aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db01498679f74da4bf4beedf328f5da2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:58.918127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6372b725-27ef-4784-be0c-256de46f85aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db01498679f74da4bf4beedf328f5da2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:58.918796 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6372b725-27ef-4784-be0c-256de46f85aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db01498679f74da4bf4beedf328f5da2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:58.926981 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6372b725-27ef-4784-be0c-256de46f85aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db01498679f74da4bf4beedf328f5da2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:58.938738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6372b725-27ef-4784-be0c-256de46f85aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db01498679f74da4bf4beedf328f5da2'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:59.004544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1258/2545] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:48:58 2022] DELETE /identity/v3/users/db01498679f74da4bf4beedf328f5da2 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:59.232662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3aaad3-2bf2-4d3f-a7f6-0f654cc6ced6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:59.237848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3aaad3-2bf2-4d3f-a7f6-0f654cc6ced6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:59.237848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3aaad3-2bf2-4d3f-a7f6-0f654cc6ced6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:59.254930 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1c3aaad3-2bf2-4d3f-a7f6-0f654cc6ced6 None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:59.256145 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1c3aaad3-2bf2-4d3f-a7f6-0f654cc6ced6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:59.321894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1288/2546] 10.222.0.64 () {58 vars in 1009 bytes} [Mon May 23 15:48:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:59.335611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372b725-27ef-4784-be0c-256de46f85aa admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:59.335611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6372b725-27ef-4784-be0c-256de46f85aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:59.340462 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3aaad3-2bf2-4d3f-a7f6-0f654cc6ced6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:59.342503 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1c3aaad3-2bf2-4d3f-a7f6-0f654cc6ced6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:59.434872 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:59.443406 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372b725-27ef-4784-be0c-256de46f85aa admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:59.453671 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b165f-67e1-4f80-a70c-91139d6680d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:59.455410 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b165f-67e1-4f80-a70c-91139d6680d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:59.456655 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b165f-67e1-4f80-a70c-91139d6680d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:59.458263 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b165f-67e1-4f80-a70c-91139d6680d0 admin admin] PATH_INFO: `/v3/projects/9aaa48a2ec82422a8eb82cce558a778a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:59.466196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b165f-67e1-4f80-a70c-91139d6680d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aaa48a2ec82422a8eb82cce558a778a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:59.472786 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b165f-67e1-4f80-a70c-91139d6680d0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:59.485460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3aaad3-2bf2-4d3f-a7f6-0f654cc6ced6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:59.492508 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0492c-6216-4d2d-85b4-62b40279e93c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:59.493505 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0492c-6216-4d2d-85b4-62b40279e93c service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:59.494227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0492c-6216-4d2d-85b4-62b40279e93c service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:59.494832 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0492c-6216-4d2d-85b4-62b40279e93c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:59.504815 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0492c-6216-4d2d-85b4-62b40279e93c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:59.506922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0492c-6216-4d2d-85b4-62b40279e93c service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:59.515923 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1289/2547] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:48:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 15:48:59.516878 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:59.605405 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-691b165f-67e1-4f80-a70c-91139d6680d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aaa48a2ec82422a8eb82cce558a778a', 'request_id': 'req-691b165f-67e1-4f80-a70c-91139d6680d0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:59.611091 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-691b165f-67e1-4f80-a70c-91139d6680d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aaa48a2ec82422a8eb82cce558a778a', 'request_id': 'req-691b165f-67e1-4f80-a70c-91139d6680d0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:59.618303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-691b165f-67e1-4f80-a70c-91139d6680d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aaa48a2ec82422a8eb82cce558a778a', 'request_id': 'req-691b165f-67e1-4f80-a70c-91139d6680d0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:59.726803 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-691b165f-67e1-4f80-a70c-91139d6680d0 admin admin] The token cache is being invalidate because project 9aaa48a2ec82422a8eb82cce558a778a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:48:59.734546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-691b165f-67e1-4f80-a70c-91139d6680d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:59.734546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-691b165f-67e1-4f80-a70c-91139d6680d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:48:59.734546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1259/2548] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:48:59 2022] DELETE /identity/v3/projects/9aaa48a2ec82422a8eb82cce558a778a => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:48:59.735157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b12f8b-73e1-4724-a50d-2b7a162a6e70 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:59.735157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b12f8b-73e1-4724-a50d-2b7a162a6e70 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:59.735420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b12f8b-73e1-4724-a50d-2b7a162a6e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:59.802807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-10b12f8b-73e1-4724-a50d-2b7a162a6e70 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:59.804704 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-10b12f8b-73e1-4724-a50d-2b7a162a6e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:59.900543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1290/2549] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:48:59.905872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b9f36-7161-48f3-931b-fe63d4a35e77 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:59.906132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b9f36-7161-48f3-931b-fe63d4a35e77 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:59.906606 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b9f36-7161-48f3-931b-fe63d4a35e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:59.946394 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c11b9f36-7161-48f3-931b-fe63d4a35e77 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:59.949109 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c11b9f36-7161-48f3-931b-fe63d4a35e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:59.993328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1260/2550] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:48:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:00.001545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b12f8b-73e1-4724-a50d-2b7a162a6e70 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:00.022193 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b12f8b-73e1-4724-a50d-2b7a162a6e70 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:00.028015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645576f2-429d-42f5-ab7a-547724b42706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:00.034880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645576f2-429d-42f5-ab7a-547724b42706 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:00.034880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645576f2-429d-42f5-ab7a-547724b42706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:00.034880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645576f2-429d-42f5-ab7a-547724b42706 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:00.034880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645576f2-429d-42f5-ab7a-547724b42706 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:49:00.034880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645576f2-429d-42f5-ab7a-547724b42706 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:00.034880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645576f2-429d-42f5-ab7a-547724b42706 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:00.077133 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1291/2551] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:49:00.079279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b9f36-7161-48f3-931b-fe63d4a35e77 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:00.091707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b9f36-7161-48f3-931b-fe63d4a35e77 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:00.101500 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb226e4-9d21-4f46-a841-4f0f6304173d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:00.108048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb226e4-9d21-4f46-a841-4f0f6304173d admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:00.108435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb226e4-9d21-4f46-a841-4f0f6304173d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:00.108794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb226e4-9d21-4f46-a841-4f0f6304173d admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:00.109723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb226e4-9d21-4f46-a841-4f0f6304173d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:00.111891 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb226e4-9d21-4f46-a841-4f0f6304173d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:00.217734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1261/2552] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:49:00 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:49:00.221685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645576f2-429d-42f5-ab7a-547724b42706 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:00.240966 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645576f2-429d-42f5-ab7a-547724b42706 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:00.331257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23d0c0-bf0f-4122-953e-d4b93e9453f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:00.331990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23d0c0-bf0f-4122-953e-d4b93e9453f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:00.332281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23d0c0-bf0f-4122-953e-d4b93e9453f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:00.332592 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23d0c0-bf0f-4122-953e-d4b93e9453f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:00.333337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23d0c0-bf0f-4122-953e-d4b93e9453f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:00.335519 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23d0c0-bf0f-4122-953e-d4b93e9453f9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:00.467119 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1292/2553] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:49:00 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:49:00.468480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb226e4-9d21-4f46-a841-4f0f6304173d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:00.490573 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb226e4-9d21-4f46-a841-4f0f6304173d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:00.504697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1552f5-2695-4ed4-83c5-7fbde47e94de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:00.505431 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1552f5-2695-4ed4-83c5-7fbde47e94de admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:00.505724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1552f5-2695-4ed4-83c5-7fbde47e94de admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:00.506009 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1552f5-2695-4ed4-83c5-7fbde47e94de admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:00.506761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1552f5-2695-4ed4-83c5-7fbde47e94de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:00.510824 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1552f5-2695-4ed4-83c5-7fbde47e94de admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:00.529674 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1262/2554] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:49:00.534166 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23d0c0-bf0f-4122-953e-d4b93e9453f9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:00.568325 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23d0c0-bf0f-4122-953e-d4b93e9453f9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:00.585088 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbf80e-0491-4b9b-8c01-961302b54f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:00.586806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbf80e-0491-4b9b-8c01-961302b54f82 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:00.587263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbf80e-0491-4b9b-8c01-961302b54f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:00.588875 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbf80e-0491-4b9b-8c01-961302b54f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:00.589845 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbf80e-0491-4b9b-8c01-961302b54f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:00.598382 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbf80e-0491-4b9b-8c01-961302b54f82 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:00.623825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1293/2555] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:49:00.624996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1552f5-2695-4ed4-83c5-7fbde47e94de admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:00.641314 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1552f5-2695-4ed4-83c5-7fbde47e94de admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:00.658838 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac75f5-4a1b-4c43-900c-858f4b94101a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:00.659835 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac75f5-4a1b-4c43-900c-858f4b94101a admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:00.660104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac75f5-4a1b-4c43-900c-858f4b94101a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:00.660392 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac75f5-4a1b-4c43-900c-858f4b94101a admin admin] PATH_INFO: `/v3/projects/d92074a57c1f44289cbc29b9d18fc12a/users/89985759a02f4a7aaa463f60dba6391c/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:00.692495 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac75f5-4a1b-4c43-900c-858f4b94101a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d92074a57c1f44289cbc29b9d18fc12a, user_id=89985759a02f4a7aaa463f60dba6391c, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:00.695266 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac75f5-4a1b-4c43-900c-858f4b94101a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:00.771812 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1263/2556] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:49:00 2022] PUT /identity/v3/projects/d92074a57c1f44289cbc29b9d18fc12a/users/89985759a02f4a7aaa463f60dba6391c/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:00.784996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee27f1-9ac2-4648-8418-4faaef39bb2e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:00.785357 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee27f1-9ac2-4648-8418-4faaef39bb2e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:00.785648 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee27f1-9ac2-4648-8418-4faaef39bb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:00.830489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1dee27f1-9ac2-4648-8418-4faaef39bb2e None None] MFA Rules not processed for user `89985759a02f4a7aaa463f60dba6391c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:00.832246 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1dee27f1-9ac2-4648-8418-4faaef39bb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:00.913601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:00.989732 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1294/2557] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:01.002323 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac75f5-4a1b-4c43-900c-858f4b94101a admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:01.005594 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-52ac75f5-4a1b-4c43-900c-858f4b94101a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:01.078022 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac75f5-4a1b-4c43-900c-858f4b94101a admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:01.083159 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f5cac-c33b-44a9-8af9-32f262ed4bda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:01.084561 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f5cac-c33b-44a9-8af9-32f262ed4bda service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:01.085298 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f5cac-c33b-44a9-8af9-32f262ed4bda service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:01.086010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f5cac-c33b-44a9-8af9-32f262ed4bda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:01.096079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f5cac-c33b-44a9-8af9-32f262ed4bda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:01.098599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f5cac-c33b-44a9-8af9-32f262ed4bda service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:01.114861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1264/2558] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:49:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:02.999953 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee27f1-9ac2-4648-8418-4faaef39bb2e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:03.001861 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1dee27f1-9ac2-4648-8418-4faaef39bb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:03.105919 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee27f1-9ac2-4648-8418-4faaef39bb2e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:03.110906 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b94fd5-3b73-4103-bea3-cdb647eb0bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:03.111915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b94fd5-3b73-4103-bea3-cdb647eb0bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:03.112312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b94fd5-3b73-4103-bea3-cdb647eb0bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:03.113299 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b94fd5-3b73-4103-bea3-cdb647eb0bef admin admin] PATH_INFO: `/v3/users/8b983c0a2a124ac3aa3403e0777612f7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:03.114912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b94fd5-3b73-4103-bea3-cdb647eb0bef admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b983c0a2a124ac3aa3403e0777612f7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:03.116966 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b94fd5-3b73-4103-bea3-cdb647eb0bef admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:03.360269 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a7b94fd5-3b73-4103-bea3-cdb647eb0bef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b983c0a2a124ac3aa3403e0777612f7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:03.383332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a7b94fd5-3b73-4103-bea3-cdb647eb0bef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b983c0a2a124ac3aa3403e0777612f7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:03.444609 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a7b94fd5-3b73-4103-bea3-cdb647eb0bef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b983c0a2a124ac3aa3403e0777612f7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:03.445429 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a7b94fd5-3b73-4103-bea3-cdb647eb0bef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b983c0a2a124ac3aa3403e0777612f7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:03.453647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a7b94fd5-3b73-4103-bea3-cdb647eb0bef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b983c0a2a124ac3aa3403e0777612f7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:03.462403 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1295/2559] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:49:02 2022] DELETE /identity/v3/users/8b983c0a2a124ac3aa3403e0777612f7 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:03.794929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f5cac-c33b-44a9-8af9-32f262ed4bda service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:03.833657 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f5cac-c33b-44a9-8af9-32f262ed4bda service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:04.012136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:04.025770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929daa21-2a56-4027-8c49-cc1ca6186e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:04.027628 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929daa21-2a56-4027-8c49-cc1ca6186e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:04.027919 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929daa21-2a56-4027-8c49-cc1ca6186e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:04.028205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929daa21-2a56-4027-8c49-cc1ca6186e41 admin admin] PATH_INFO: `/v3/projects/e0860b3d072246d49cd43a900351d571` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:04.029601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929daa21-2a56-4027-8c49-cc1ca6186e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0860b3d072246d49cd43a900351d571)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:04.033610 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929daa21-2a56-4027-8c49-cc1ca6186e41 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:04.227783 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-929daa21-2a56-4027-8c49-cc1ca6186e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0860b3d072246d49cd43a900351d571', 'request_id': 'req-929daa21-2a56-4027-8c49-cc1ca6186e41'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:04.234044 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-929daa21-2a56-4027-8c49-cc1ca6186e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0860b3d072246d49cd43a900351d571', 'request_id': 'req-929daa21-2a56-4027-8c49-cc1ca6186e41'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:04.253703 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-929daa21-2a56-4027-8c49-cc1ca6186e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0860b3d072246d49cd43a900351d571', 'request_id': 'req-929daa21-2a56-4027-8c49-cc1ca6186e41'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:04.330644 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-929daa21-2a56-4027-8c49-cc1ca6186e41 admin admin] The token cache is being invalidate because project e0860b3d072246d49cd43a900351d571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:49:04.331592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-929daa21-2a56-4027-8c49-cc1ca6186e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:04.332713 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-929daa21-2a56-4027-8c49-cc1ca6186e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:04.334795 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1265/2560] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:49:03 2022] DELETE /identity/v3/projects/e0860b3d072246d49cd43a900351d571 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:04.342504 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b94fd5-3b73-4103-bea3-cdb647eb0bef admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:04.344797 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a7b94fd5-3b73-4103-bea3-cdb647eb0bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:04.362489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b94fd5-3b73-4103-bea3-cdb647eb0bef admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:04.455497 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:04.463950 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d3efd-42a3-4a48-9a3f-bf7242d9bfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:04.465010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d3efd-42a3-4a48-9a3f-bf7242d9bfce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:04.465470 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d3efd-42a3-4a48-9a3f-bf7242d9bfce admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:04.465883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d3efd-42a3-4a48-9a3f-bf7242d9bfce admin admin] PATH_INFO: `/v3/users/16a831b346e94d79925d017711a07417` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:04.467171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d3efd-42a3-4a48-9a3f-bf7242d9bfce admin admin] RBAC: Authorizing `identity:delete_user(user_id=16a831b346e94d79925d017711a07417)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:04.469344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d3efd-42a3-4a48-9a3f-bf7242d9bfce admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:04.666136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-472d3efd-42a3-4a48-9a3f-bf7242d9bfce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16a831b346e94d79925d017711a07417'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:04.680953 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-472d3efd-42a3-4a48-9a3f-bf7242d9bfce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16a831b346e94d79925d017711a07417'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:04.758937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-472d3efd-42a3-4a48-9a3f-bf7242d9bfce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16a831b346e94d79925d017711a07417'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:04.759544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-472d3efd-42a3-4a48-9a3f-bf7242d9bfce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16a831b346e94d79925d017711a07417'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:04.776031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-472d3efd-42a3-4a48-9a3f-bf7242d9bfce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16a831b346e94d79925d017711a07417'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:04.808132 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1296/2561] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:49:04 2022] DELETE /identity/v3/users/16a831b346e94d79925d017711a07417 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:05.145258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929daa21-2a56-4027-8c49-cc1ca6186e41 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:05.147392 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-929daa21-2a56-4027-8c49-cc1ca6186e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:05.167917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929daa21-2a56-4027-8c49-cc1ca6186e41 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:05.296659 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a616d1-69a0-4c35-bc3c-bcf11c2f9083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:05.298482 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a616d1-69a0-4c35-bc3c-bcf11c2f9083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:05.298986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a616d1-69a0-4c35-bc3c-bcf11c2f9083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:05.299756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a616d1-69a0-4c35-bc3c-bcf11c2f9083 admin admin] PATH_INFO: `/v3/projects/8e7b1014ab1b4cb58f1c556dd952271d` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:05.301590 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a616d1-69a0-4c35-bc3c-bcf11c2f9083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e7b1014ab1b4cb58f1c556dd952271d)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:05.304853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a616d1-69a0-4c35-bc3c-bcf11c2f9083 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:05.470498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f2a616d1-69a0-4c35-bc3c-bcf11c2f9083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e7b1014ab1b4cb58f1c556dd952271d', 'request_id': 'req-f2a616d1-69a0-4c35-bc3c-bcf11c2f9083'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:05.479503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f2a616d1-69a0-4c35-bc3c-bcf11c2f9083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e7b1014ab1b4cb58f1c556dd952271d', 'request_id': 'req-f2a616d1-69a0-4c35-bc3c-bcf11c2f9083'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:05.495633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f2a616d1-69a0-4c35-bc3c-bcf11c2f9083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e7b1014ab1b4cb58f1c556dd952271d', 'request_id': 'req-f2a616d1-69a0-4c35-bc3c-bcf11c2f9083'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:05.616953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f2a616d1-69a0-4c35-bc3c-bcf11c2f9083 admin admin] The token cache is being invalidate because project 8e7b1014ab1b4cb58f1c556dd952271d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:49:05.617310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f2a616d1-69a0-4c35-bc3c-bcf11c2f9083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:05.617982 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f2a616d1-69a0-4c35-bc3c-bcf11c2f9083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:05.621369 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1266/2562] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:49:05 2022] DELETE /identity/v3/projects/8e7b1014ab1b4cb58f1c556dd952271d => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:05.841750 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d3efd-42a3-4a48-9a3f-bf7242d9bfce admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:05.844310 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-472d3efd-42a3-4a48-9a3f-bf7242d9bfce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:05.873544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d3efd-42a3-4a48-9a3f-bf7242d9bfce admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:06.031496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa0990-b2ac-4475-8f24-5e8334b7b363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:06.031496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa0990-b2ac-4475-8f24-5e8334b7b363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:06.031496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa0990-b2ac-4475-8f24-5e8334b7b363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:06.031496 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa0990-b2ac-4475-8f24-5e8334b7b363 admin admin] PATH_INFO: `/v3/users/89985759a02f4a7aaa463f60dba6391c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:06.032015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa0990-b2ac-4475-8f24-5e8334b7b363 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89985759a02f4a7aaa463f60dba6391c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:06.032015 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa0990-b2ac-4475-8f24-5e8334b7b363 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:07.115825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-59fa0990-b2ac-4475-8f24-5e8334b7b363 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89985759a02f4a7aaa463f60dba6391c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:07.130956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-59fa0990-b2ac-4475-8f24-5e8334b7b363 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89985759a02f4a7aaa463f60dba6391c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:07.256470 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-59fa0990-b2ac-4475-8f24-5e8334b7b363 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89985759a02f4a7aaa463f60dba6391c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:07.257272 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-59fa0990-b2ac-4475-8f24-5e8334b7b363 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89985759a02f4a7aaa463f60dba6391c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:07.262864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:07.266311 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-59fa0990-b2ac-4475-8f24-5e8334b7b363 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89985759a02f4a7aaa463f60dba6391c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:07.274268 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1297/2563] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:49:05 2022] DELETE /identity/v3/users/89985759a02f4a7aaa463f60dba6391c => generated 0 bytes in 1432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:07.277636 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7985d-dd2a-489b-89f5-29ba90141745 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:07.278192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7985d-dd2a-489b-89f5-29ba90141745 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:07.278192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7985d-dd2a-489b-89f5-29ba90141745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:07.298417 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-5eb7985d-dd2a-489b-89f5-29ba90141745 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:07.300599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5eb7985d-dd2a-489b-89f5-29ba90141745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:07.306435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:07.382020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1267/2564] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:07.392096 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa0990-b2ac-4475-8f24-5e8334b7b363 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:07.395067 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-59fa0990-b2ac-4475-8f24-5e8334b7b363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:07.463862 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa0990-b2ac-4475-8f24-5e8334b7b363 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:07.478927 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7d776-d050-48a6-8342-b6bf7681b1d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:07.480381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7d776-d050-48a6-8342-b6bf7681b1d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:07.480779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7d776-d050-48a6-8342-b6bf7681b1d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:07.481191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7d776-d050-48a6-8342-b6bf7681b1d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:07.492646 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7d776-d050-48a6-8342-b6bf7681b1d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:07.494545 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7d776-d050-48a6-8342-b6bf7681b1d3 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:07.506333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1298/2565] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:49:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:08.121139 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7985d-dd2a-489b-89f5-29ba90141745 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:08.122662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5eb7985d-dd2a-489b-89f5-29ba90141745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:08.136518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7985d-dd2a-489b-89f5-29ba90141745 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:08.141037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fe496-e197-4fe0-a1df-12493c3aee02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:08.141927 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fe496-e197-4fe0-a1df-12493c3aee02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:08.142213 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fe496-e197-4fe0-a1df-12493c3aee02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:08.142490 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fe496-e197-4fe0-a1df-12493c3aee02 admin admin] PATH_INFO: `/v3/projects/d92074a57c1f44289cbc29b9d18fc12a` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:08.144344 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fe496-e197-4fe0-a1df-12493c3aee02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d92074a57c1f44289cbc29b9d18fc12a)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:08.146392 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fe496-e197-4fe0-a1df-12493c3aee02 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:08.263439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-387fe496-e197-4fe0-a1df-12493c3aee02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd92074a57c1f44289cbc29b9d18fc12a', 'request_id': 'req-387fe496-e197-4fe0-a1df-12493c3aee02'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:08.268989 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-387fe496-e197-4fe0-a1df-12493c3aee02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd92074a57c1f44289cbc29b9d18fc12a', 'request_id': 'req-387fe496-e197-4fe0-a1df-12493c3aee02'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:08.275134 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-387fe496-e197-4fe0-a1df-12493c3aee02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd92074a57c1f44289cbc29b9d18fc12a', 'request_id': 'req-387fe496-e197-4fe0-a1df-12493c3aee02'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:08.364309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-387fe496-e197-4fe0-a1df-12493c3aee02 admin admin] The token cache is being invalidate because project d92074a57c1f44289cbc29b9d18fc12a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:49:08.364996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-387fe496-e197-4fe0-a1df-12493c3aee02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:08.365261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-387fe496-e197-4fe0-a1df-12493c3aee02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:08.366810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1268/2566] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:49:08 2022] DELETE /identity/v3/projects/d92074a57c1f44289cbc29b9d18fc12a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:49:08.371797 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a0a7e-aa60-4458-bdf7-470eb89c1385 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:08.372321 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a0a7e-aa60-4458-bdf7-470eb89c1385 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:08.372457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a0a7e-aa60-4458-bdf7-470eb89c1385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:08.390472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-890a0a7e-aa60-4458-bdf7-470eb89c1385 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:08.391625 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-890a0a7e-aa60-4458-bdf7-470eb89c1385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:08.451831 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1299/2567] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:08.461406 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e0551-dca3-4a77-b278-00728490cbf5 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:08.461934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e0551-dca3-4a77-b278-00728490cbf5 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:08.462048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e0551-dca3-4a77-b278-00728490cbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:08.480607 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-867e0551-dca3-4a77-b278-00728490cbf5 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:08.481872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-867e0551-dca3-4a77-b278-00728490cbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:08.490776 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1269/2568] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:08.495000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a0a7e-aa60-4458-bdf7-470eb89c1385 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:08.505351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a0a7e-aa60-4458-bdf7-470eb89c1385 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:08.509827 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d90c60-2b16-4f67-8d5e-d3227b006839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:08.510479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d90c60-2b16-4f67-8d5e-d3227b006839 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:08.510765 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d90c60-2b16-4f67-8d5e-d3227b006839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:08.511041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d90c60-2b16-4f67-8d5e-d3227b006839 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:08.511760 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d90c60-2b16-4f67-8d5e-d3227b006839 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:49:08.512069 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d90c60-2b16-4f67-8d5e-d3227b006839 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:08.514054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d90c60-2b16-4f67-8d5e-d3227b006839 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:08.527329 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1300/2569] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:49:08.531718 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e0551-dca3-4a77-b278-00728490cbf5 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:08.541170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e0551-dca3-4a77-b278-00728490cbf5 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:08.546190 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cead382-4c38-42c2-8422-0096c719d103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:08.546865 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cead382-4c38-42c2-8422-0096c719d103 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:08.547157 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cead382-4c38-42c2-8422-0096c719d103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:08.547444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cead382-4c38-42c2-8422-0096c719d103 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:08.548156 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cead382-4c38-42c2-8422-0096c719d103 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:08.550789 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cead382-4c38-42c2-8422-0096c719d103 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:08.681908 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1270/2570] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:49:08 2022] POST /identity/v3/projects => generated 365 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:49:08.685716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d90c60-2b16-4f67-8d5e-d3227b006839 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:08.696355 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d90c60-2b16-4f67-8d5e-d3227b006839 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:08.756114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738f007-5d23-46d7-aa70-bdf46c71fffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:08.759631 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738f007-5d23-46d7-aa70-bdf46c71fffa admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:08.759631 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738f007-5d23-46d7-aa70-bdf46c71fffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:08.759631 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738f007-5d23-46d7-aa70-bdf46c71fffa admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:08.759631 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738f007-5d23-46d7-aa70-bdf46c71fffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:08.760010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738f007-5d23-46d7-aa70-bdf46c71fffa admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:08.838914 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1301/2571] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:49:08 2022] POST /identity/v3/users => generated 283 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:49:08.843130 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cead382-4c38-42c2-8422-0096c719d103 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:08.854599 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cead382-4c38-42c2-8422-0096c719d103 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:08.859249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bace3e9-4bc4-4dc4-934d-81c437cb0461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:08.859906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bace3e9-4bc4-4dc4-934d-81c437cb0461 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:08.860234 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bace3e9-4bc4-4dc4-934d-81c437cb0461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:08.860515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bace3e9-4bc4-4dc4-934d-81c437cb0461 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:08.861281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bace3e9-4bc4-4dc4-934d-81c437cb0461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:08.863574 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bace3e9-4bc4-4dc4-934d-81c437cb0461 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:08.874055 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1271/2572] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:49:08.878037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738f007-5d23-46d7-aa70-bdf46c71fffa admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:08.887854 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738f007-5d23-46d7-aa70-bdf46c71fffa admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:08.892490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20e456-1d35-43c6-8c5e-8c146e62ebea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:08.893255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20e456-1d35-43c6-8c5e-8c146e62ebea admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:08.893772 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20e456-1d35-43c6-8c5e-8c146e62ebea admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:08.893937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20e456-1d35-43c6-8c5e-8c146e62ebea admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:08.894659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20e456-1d35-43c6-8c5e-8c146e62ebea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:08.896250 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20e456-1d35-43c6-8c5e-8c146e62ebea admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:08.906409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1302/2573] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:49:08.910891 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bace3e9-4bc4-4dc4-934d-81c437cb0461 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:08.920542 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bace3e9-4bc4-4dc4-934d-81c437cb0461 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:08.926032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684e541-d439-4784-8919-b1c1e9dc706f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:08.926899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684e541-d439-4784-8919-b1c1e9dc706f admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:08.927185 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684e541-d439-4784-8919-b1c1e9dc706f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:08.927461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684e541-d439-4784-8919-b1c1e9dc706f admin admin] PATH_INFO: `/v3/projects/a9e404c07902411090953743248f1551/users/3452984e01354130828ac96326821503/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:08.942004 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684e541-d439-4784-8919-b1c1e9dc706f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e404c07902411090953743248f1551, user_id=3452984e01354130828ac96326821503, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:08.944710 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684e541-d439-4784-8919-b1c1e9dc706f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:09.010653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1272/2574] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:49:08 2022] PUT /identity/v3/projects/a9e404c07902411090953743248f1551/users/3452984e01354130828ac96326821503/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:09.022288 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b7fdd-5e06-4334-830e-aa3d3656b1f8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:09.023190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b7fdd-5e06-4334-830e-aa3d3656b1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:09.023190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b7fdd-5e06-4334-830e-aa3d3656b1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:09.028903 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:09.059495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ac4b7fdd-5e06-4334-830e-aa3d3656b1f8 None None] MFA Rules not processed for user `3452984e01354130828ac96326821503`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:09.059747 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ac4b7fdd-5e06-4334-830e-aa3d3656b1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:09.141691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1303/2575] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:09.153525 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684e541-d439-4784-8919-b1c1e9dc706f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:09.155293 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1684e541-d439-4784-8919-b1c1e9dc706f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:09.191280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:09.214676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684e541-d439-4784-8919-b1c1e9dc706f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:09.218237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f6915-a6f4-4c3e-be70-aa2e3d6bfd69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:09.218922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f6915-a6f4-4c3e-be70-aa2e3d6bfd69 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:09.219203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f6915-a6f4-4c3e-be70-aa2e3d6bfd69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:09.219482 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f6915-a6f4-4c3e-be70-aa2e3d6bfd69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:09.228648 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f6915-a6f4-4c3e-be70-aa2e3d6bfd69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:09.230193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f6915-a6f4-4c3e-be70-aa2e3d6bfd69 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:09.239125 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1273/2576] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:49:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:19.138485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=31 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:19.764321 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=12 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:22.648896 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13024a4a-c14b-450c-bffe-96c5e795f4bd None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:22.649509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13024a4a-c14b-450c-bffe-96c5e795f4bd None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:22.649509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13024a4a-c14b-450c-bffe-96c5e795f4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:22.671459 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-13024a4a-c14b-450c-bffe-96c5e795f4bd None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:22.672689 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-13024a4a-c14b-450c-bffe-96c5e795f4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:22.736698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1304/2577] 10.222.0.64 () {58 vars in 967 bytes} [Mon May 23 15:49:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:22.750518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f6915-a6f4-4c3e-be70-aa2e3d6bfd69 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:22.752739 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-bc1f6915-a6f4-4c3e-be70-aa2e3d6bfd69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:22.846692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f6915-a6f4-4c3e-be70-aa2e3d6bfd69 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:22.852572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31d38f-0bd5-40fd-9c05-86aebcaefb8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:22.853435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31d38f-0bd5-40fd-9c05-86aebcaefb8e service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:22.853803 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31d38f-0bd5-40fd-9c05-86aebcaefb8e service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:22.856653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31d38f-0bd5-40fd-9c05-86aebcaefb8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:22.866631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31d38f-0bd5-40fd-9c05-86aebcaefb8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:22.868399 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31d38f-0bd5-40fd-9c05-86aebcaefb8e service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:22.880006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1274/2578] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:49:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:49:32.729692 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=25 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:32.875227 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=32 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:33.239152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13024a4a-c14b-450c-bffe-96c5e795f4bd None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:33.240561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-13024a4a-c14b-450c-bffe-96c5e795f4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:33.298716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13024a4a-c14b-450c-bffe-96c5e795f4bd None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:33.302709 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3abd5-16cb-4b0f-9184-699ac53b3a28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:33.303363 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3abd5-16cb-4b0f-9184-699ac53b3a28 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:33.303651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3abd5-16cb-4b0f-9184-699ac53b3a28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:33.303933 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3abd5-16cb-4b0f-9184-699ac53b3a28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:33.305735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-54a3abd5-16cb-4b0f-9184-699ac53b3a28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:33.372612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3abd5-16cb-4b0f-9184-699ac53b3a28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:33.374125 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3abd5-16cb-4b0f-9184-699ac53b3a28 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:33.384084 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1305/2579] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:49:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:37.153415 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31d38f-0bd5-40fd-9c05-86aebcaefb8e service swift] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:37.154959 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2f31d38f-0bd5-40fd-9c05-86aebcaefb8e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:37.169088 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31d38f-0bd5-40fd-9c05-86aebcaefb8e service swift] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:37.173572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c675b9-8349-4e87-83fc-19a506fb2ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:37.174427 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c675b9-8349-4e87-83fc-19a506fb2ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:37.174979 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c675b9-8349-4e87-83fc-19a506fb2ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:37.175256 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c675b9-8349-4e87-83fc-19a506fb2ddb admin admin] PATH_INFO: `/v3/users/472049224bbf48699b8a9410ec61aa9e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:37.177003 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c675b9-8349-4e87-83fc-19a506fb2ddb admin admin] RBAC: Authorizing `identity:delete_user(user_id=472049224bbf48699b8a9410ec61aa9e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:37.178945 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c675b9-8349-4e87-83fc-19a506fb2ddb admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:37.481647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d3c675b9-8349-4e87-83fc-19a506fb2ddb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '472049224bbf48699b8a9410ec61aa9e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:37.482315 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d3c675b9-8349-4e87-83fc-19a506fb2ddb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '472049224bbf48699b8a9410ec61aa9e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:37.482920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d3c675b9-8349-4e87-83fc-19a506fb2ddb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '472049224bbf48699b8a9410ec61aa9e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:37.490588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d3c675b9-8349-4e87-83fc-19a506fb2ddb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '472049224bbf48699b8a9410ec61aa9e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:37.502291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d3c675b9-8349-4e87-83fc-19a506fb2ddb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '472049224bbf48699b8a9410ec61aa9e'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:37.684965 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1275/2580] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:49:37 2022] DELETE /identity/v3/users/472049224bbf48699b8a9410ec61aa9e => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:38.148973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3abd5-16cb-4b0f-9184-699ac53b3a28 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:38.163020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3abd5-16cb-4b0f-9184-699ac53b3a28 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:38.218568 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f57e94-7a8b-422c-b9c9-ce089f55ff2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:38.220621 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f57e94-7a8b-422c-b9c9-ce089f55ff2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:38.220867 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f57e94-7a8b-422c-b9c9-ce089f55ff2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:38.221171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f57e94-7a8b-422c-b9c9-ce089f55ff2e admin admin] PATH_INFO: `/v3/projects/a15ae9d15a64468f80628ed51c2e42d6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:38.222445 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f57e94-7a8b-422c-b9c9-ce089f55ff2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a15ae9d15a64468f80628ed51c2e42d6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:38.224990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f57e94-7a8b-422c-b9c9-ce089f55ff2e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:38.382801 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-32f57e94-7a8b-422c-b9c9-ce089f55ff2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a15ae9d15a64468f80628ed51c2e42d6', 'request_id': 'req-32f57e94-7a8b-422c-b9c9-ce089f55ff2e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:38.384674 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:38.389261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-32f57e94-7a8b-422c-b9c9-ce089f55ff2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a15ae9d15a64468f80628ed51c2e42d6', 'request_id': 'req-32f57e94-7a8b-422c-b9c9-ce089f55ff2e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:38.472396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-32f57e94-7a8b-422c-b9c9-ce089f55ff2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a15ae9d15a64468f80628ed51c2e42d6', 'request_id': 'req-32f57e94-7a8b-422c-b9c9-ce089f55ff2e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:38.483971 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-32f57e94-7a8b-422c-b9c9-ce089f55ff2e admin admin] The token cache is being invalidate because project a15ae9d15a64468f80628ed51c2e42d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:49:38.484266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-32f57e94-7a8b-422c-b9c9-ce089f55ff2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:38.484898 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-32f57e94-7a8b-422c-b9c9-ce089f55ff2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:38.486495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1306/2581] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:49:38 2022] DELETE /identity/v3/projects/a15ae9d15a64468f80628ed51c2e42d6 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:38.492230 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccd7af-176e-4971-936f-38be3dd020e6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:38.492758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccd7af-176e-4971-936f-38be3dd020e6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:38.492886 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccd7af-176e-4971-936f-38be3dd020e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:38.511546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-54ccd7af-176e-4971-936f-38be3dd020e6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:38.512770 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-54ccd7af-176e-4971-936f-38be3dd020e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:38.580754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1276/2582] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:38.588435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2af8c-2a4b-400e-b3b2-716cd1c514c2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:38.588728 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2af8c-2a4b-400e-b3b2-716cd1c514c2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:38.589007 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2af8c-2a4b-400e-b3b2-716cd1c514c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:38.607675 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-11d2af8c-2a4b-400e-b3b2-716cd1c514c2 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:38.610202 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-11d2af8c-2a4b-400e-b3b2-716cd1c514c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:38.617561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1307/2583] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:38.621736 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccd7af-176e-4971-936f-38be3dd020e6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:38.631980 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccd7af-176e-4971-936f-38be3dd020e6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:38.636539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cb1be-ca32-425d-87ff-562082e6956a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:38.644899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cb1be-ca32-425d-87ff-562082e6956a admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:38.644899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cb1be-ca32-425d-87ff-562082e6956a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:38.644899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cb1be-ca32-425d-87ff-562082e6956a admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:38.644899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cb1be-ca32-425d-87ff-562082e6956a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:49:38.644899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cb1be-ca32-425d-87ff-562082e6956a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:38.644899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cb1be-ca32-425d-87ff-562082e6956a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:38.651675 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1277/2584] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:49:38.655957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2af8c-2a4b-400e-b3b2-716cd1c514c2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:38.665691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2af8c-2a4b-400e-b3b2-716cd1c514c2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:38.670235 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9de6c4-dc2f-46b7-9387-ea96bc451f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:38.670880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9de6c4-dc2f-46b7-9387-ea96bc451f10 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:38.671167 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9de6c4-dc2f-46b7-9387-ea96bc451f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:38.671446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9de6c4-dc2f-46b7-9387-ea96bc451f10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:38.672132 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9de6c4-dc2f-46b7-9387-ea96bc451f10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:38.674059 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9de6c4-dc2f-46b7-9387-ea96bc451f10 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:38.771574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1308/2585] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:49:38 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:49:38.775982 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cb1be-ca32-425d-87ff-562082e6956a admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:38.787267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cb1be-ca32-425d-87ff-562082e6956a admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:38.842344 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625f1ec-ad92-4938-93cf-2be45aa32fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:38.842969 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625f1ec-ad92-4938-93cf-2be45aa32fef admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:38.843257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625f1ec-ad92-4938-93cf-2be45aa32fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:38.843557 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625f1ec-ad92-4938-93cf-2be45aa32fef admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:38.844236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625f1ec-ad92-4938-93cf-2be45aa32fef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:38.846189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625f1ec-ad92-4938-93cf-2be45aa32fef admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:38.853512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:38.952664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1278/2586] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:49:38 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:49:38.956809 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9de6c4-dc2f-46b7-9387-ea96bc451f10 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:38.968488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9de6c4-dc2f-46b7-9387-ea96bc451f10 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:38.973975 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51483d-6a83-4674-85c8-3a6f9b9c2688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:38.974646 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51483d-6a83-4674-85c8-3a6f9b9c2688 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:38.974930 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51483d-6a83-4674-85c8-3a6f9b9c2688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:38.975201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51483d-6a83-4674-85c8-3a6f9b9c2688 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:38.975916 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51483d-6a83-4674-85c8-3a6f9b9c2688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:38.977514 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51483d-6a83-4674-85c8-3a6f9b9c2688 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:38.988052 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1309/2587] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:49:38.994559 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625f1ec-ad92-4938-93cf-2be45aa32fef admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:39.004799 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625f1ec-ad92-4938-93cf-2be45aa32fef admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:39.009518 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2fa81-a936-46b1-8aff-087877c3d6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:39.014871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2fa81-a936-46b1-8aff-087877c3d6ee admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:39.014871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2fa81-a936-46b1-8aff-087877c3d6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:39.014871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2fa81-a936-46b1-8aff-087877c3d6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:39.014871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2fa81-a936-46b1-8aff-087877c3d6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:39.014871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2fa81-a936-46b1-8aff-087877c3d6ee admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:39.024140 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1279/2588] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:49:39.028389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51483d-6a83-4674-85c8-3a6f9b9c2688 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:39.039444 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51483d-6a83-4674-85c8-3a6f9b9c2688 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:39.044144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd4006-c0eb-40df-b346-83d87677ba9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:39.045023 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd4006-c0eb-40df-b346-83d87677ba9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:39.045339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd4006-c0eb-40df-b346-83d87677ba9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:39.045621 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd4006-c0eb-40df-b346-83d87677ba9b admin admin] PATH_INFO: `/v3/projects/afcda3ba1e7e4a8cb7a6f5cda10e2ac2/users/4f720cc98ea242279b7c560ae0205efd/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:39.060414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd4006-c0eb-40df-b346-83d87677ba9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=afcda3ba1e7e4a8cb7a6f5cda10e2ac2, user_id=4f720cc98ea242279b7c560ae0205efd, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:39.063094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd4006-c0eb-40df-b346-83d87677ba9b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:39.106723 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1310/2589] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:49:39 2022] PUT /identity/v3/projects/afcda3ba1e7e4a8cb7a6f5cda10e2ac2/users/4f720cc98ea242279b7c560ae0205efd/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:39.117873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffe4d7-f3ff-4361-b704-7db8025889b6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:39.118170 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffe4d7-f3ff-4361-b704-7db8025889b6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:39.118496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffe4d7-f3ff-4361-b704-7db8025889b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:39.143763 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a8ffe4d7-f3ff-4361-b704-7db8025889b6 None None] MFA Rules not processed for user `4f720cc98ea242279b7c560ae0205efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:39.145793 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a8ffe4d7-f3ff-4361-b704-7db8025889b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:39.210162 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1280/2590] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:39.221335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd4006-c0eb-40df-b346-83d87677ba9b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:39.223280 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f1fd4006-c0eb-40df-b346-83d87677ba9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:39.303398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd4006-c0eb-40df-b346-83d87677ba9b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:39.308924 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4860bb-96f0-4a0a-b5fe-13f4a96bbda5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:39.309616 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4860bb-96f0-4a0a-b5fe-13f4a96bbda5 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:39.309923 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4860bb-96f0-4a0a-b5fe-13f4a96bbda5 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:39.310192 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4860bb-96f0-4a0a-b5fe-13f4a96bbda5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:39.318289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4860bb-96f0-4a0a-b5fe-13f4a96bbda5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:39.319922 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4860bb-96f0-4a0a-b5fe-13f4a96bbda5 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:39.330414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1311/2591] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:49:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:42.455744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffe4d7-f3ff-4361-b704-7db8025889b6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:42.518494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffe4d7-f3ff-4361-b704-7db8025889b6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:42.523688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709b1f1-a5b9-4a4d-8875-d2943a42ce9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:42.524554 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709b1f1-a5b9-4a4d-8875-d2943a42ce9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:42.524845 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709b1f1-a5b9-4a4d-8875-d2943a42ce9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:42.525152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709b1f1-a5b9-4a4d-8875-d2943a42ce9b admin admin] PATH_INFO: `/v3/users/4f720cc98ea242279b7c560ae0205efd` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:42.526339 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709b1f1-a5b9-4a4d-8875-d2943a42ce9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f720cc98ea242279b7c560ae0205efd)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:42.528289 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709b1f1-a5b9-4a4d-8875-d2943a42ce9b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:42.714651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f709b1f1-a5b9-4a4d-8875-d2943a42ce9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f720cc98ea242279b7c560ae0205efd'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:42.715310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f709b1f1-a5b9-4a4d-8875-d2943a42ce9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f720cc98ea242279b7c560ae0205efd'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:42.715923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f709b1f1-a5b9-4a4d-8875-d2943a42ce9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f720cc98ea242279b7c560ae0205efd'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:42.723870 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f709b1f1-a5b9-4a4d-8875-d2943a42ce9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f720cc98ea242279b7c560ae0205efd'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:42.733931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:42.738553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-f709b1f1-a5b9-4a4d-8875-d2943a42ce9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f720cc98ea242279b7c560ae0205efd'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:42.820734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1281/2592] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:49:42 2022] DELETE /identity/v3/users/4f720cc98ea242279b7c560ae0205efd => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:42.825335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e350c7-6195-4ff7-ada6-feebe390cab1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:42.825835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e350c7-6195-4ff7-ada6-feebe390cab1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:42.825954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e350c7-6195-4ff7-ada6-feebe390cab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:42.844282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-45e350c7-6195-4ff7-ada6-feebe390cab1 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:42.847434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-45e350c7-6195-4ff7-ada6-feebe390cab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:42.905350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1312/2593] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:42.916090 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709b1f1-a5b9-4a4d-8875-d2943a42ce9b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:42.917521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f709b1f1-a5b9-4a4d-8875-d2943a42ce9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:42.975594 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709b1f1-a5b9-4a4d-8875-d2943a42ce9b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:42.979122 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f2b1c-e238-4289-88b6-f4e10efff9a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:42.979766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f2b1c-e238-4289-88b6-f4e10efff9a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:42.980055 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f2b1c-e238-4289-88b6-f4e10efff9a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:42.980336 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f2b1c-e238-4289-88b6-f4e10efff9a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:42.989606 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f2b1c-e238-4289-88b6-f4e10efff9a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:42.991048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f2b1c-e238-4289-88b6-f4e10efff9a6 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:43.000289 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1282/2594] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:49:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:43.411391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e350c7-6195-4ff7-ada6-feebe390cab1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:43.412996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-45e350c7-6195-4ff7-ada6-feebe390cab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:43.423562 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:43.429224 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e350c7-6195-4ff7-ada6-feebe390cab1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:43.435258 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ead4b7-d4b5-47b7-b6f0-f88ec66afe54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:43.436323 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ead4b7-d4b5-47b7-b6f0-f88ec66afe54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:43.436733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ead4b7-d4b5-47b7-b6f0-f88ec66afe54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:43.437165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ead4b7-d4b5-47b7-b6f0-f88ec66afe54 admin admin] PATH_INFO: `/v3/projects/afcda3ba1e7e4a8cb7a6f5cda10e2ac2` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:43.438572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ead4b7-d4b5-47b7-b6f0-f88ec66afe54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afcda3ba1e7e4a8cb7a6f5cda10e2ac2)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:43.440489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ead4b7-d4b5-47b7-b6f0-f88ec66afe54 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:43.571065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98ead4b7-d4b5-47b7-b6f0-f88ec66afe54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afcda3ba1e7e4a8cb7a6f5cda10e2ac2', 'request_id': 'req-98ead4b7-d4b5-47b7-b6f0-f88ec66afe54'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:43.580289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98ead4b7-d4b5-47b7-b6f0-f88ec66afe54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afcda3ba1e7e4a8cb7a6f5cda10e2ac2', 'request_id': 'req-98ead4b7-d4b5-47b7-b6f0-f88ec66afe54'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:43.661041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98ead4b7-d4b5-47b7-b6f0-f88ec66afe54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afcda3ba1e7e4a8cb7a6f5cda10e2ac2', 'request_id': 'req-98ead4b7-d4b5-47b7-b6f0-f88ec66afe54'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:43.671572 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98ead4b7-d4b5-47b7-b6f0-f88ec66afe54 admin admin] The token cache is being invalidate because project afcda3ba1e7e4a8cb7a6f5cda10e2ac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:49:43.672879 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98ead4b7-d4b5-47b7-b6f0-f88ec66afe54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:43.673626 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98ead4b7-d4b5-47b7-b6f0-f88ec66afe54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:43.675484 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1313/2595] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:49:43 2022] DELETE /identity/v3/projects/afcda3ba1e7e4a8cb7a6f5cda10e2ac2 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:49:43.680928 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dc283-d8b8-4332-8245-f141cb159cf4 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:43.681534 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dc283-d8b8-4332-8245-f141cb159cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:43.681676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dc283-d8b8-4332-8245-f141cb159cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:43.700643 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c00dc283-d8b8-4332-8245-f141cb159cf4 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:43.701898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c00dc283-d8b8-4332-8245-f141cb159cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:43.762536 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1283/2596] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:43.770644 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86662054-9093-4477-be11-88c4be52aa33 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:43.770940 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86662054-9093-4477-be11-88c4be52aa33 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:43.771223 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86662054-9093-4477-be11-88c4be52aa33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:43.790149 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-86662054-9093-4477-be11-88c4be52aa33 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:43.791353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-86662054-9093-4477-be11-88c4be52aa33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:43.799476 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1314/2597] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:43.804570 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dc283-d8b8-4332-8245-f141cb159cf4 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:43.814624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dc283-d8b8-4332-8245-f141cb159cf4 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:43.821029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fd62d-f1c0-4f18-8c5c-28a4cadb9ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:43.821029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fd62d-f1c0-4f18-8c5c-28a4cadb9ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:43.821029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fd62d-f1c0-4f18-8c5c-28a4cadb9ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:43.821029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fd62d-f1c0-4f18-8c5c-28a4cadb9ce4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:43.821567 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515fd62d-f1c0-4f18-8c5c-28a4cadb9ce4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:49:43.821683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515fd62d-f1c0-4f18-8c5c-28a4cadb9ce4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:43.823265 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515fd62d-f1c0-4f18-8c5c-28a4cadb9ce4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:43.835064 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1284/2598] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:49:43.839246 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86662054-9093-4477-be11-88c4be52aa33 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:43.848837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86662054-9093-4477-be11-88c4be52aa33 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:43.853486 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55709d9-1aaf-42d8-9b43-0d1053ae9cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:43.854136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55709d9-1aaf-42d8-9b43-0d1053ae9cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:43.854560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55709d9-1aaf-42d8-9b43-0d1053ae9cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:43.854759 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55709d9-1aaf-42d8-9b43-0d1053ae9cb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:43.855464 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55709d9-1aaf-42d8-9b43-0d1053ae9cb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:43.857375 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55709d9-1aaf-42d8-9b43-0d1053ae9cb3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:43.964946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1315/2599] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:49:43 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:49:43.969033 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fd62d-f1c0-4f18-8c5c-28a4cadb9ce4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:43.979476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fd62d-f1c0-4f18-8c5c-28a4cadb9ce4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:44.035579 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02c836-2edc-4ff4-b24f-2c957e748e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:44.036699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02c836-2edc-4ff4-b24f-2c957e748e8f admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:44.036987 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02c836-2edc-4ff4-b24f-2c957e748e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:44.037301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02c836-2edc-4ff4-b24f-2c957e748e8f admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:44.037997 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e02c836-2edc-4ff4-b24f-2c957e748e8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:44.039928 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e02c836-2edc-4ff4-b24f-2c957e748e8f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:44.124830 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1285/2600] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:49:43 2022] POST /identity/v3/users => generated 283 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:49:44.128151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55709d9-1aaf-42d8-9b43-0d1053ae9cb3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:44.138537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55709d9-1aaf-42d8-9b43-0d1053ae9cb3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:44.143104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96136daf-e54e-4ba9-8ade-23ec9a99924b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:44.143757 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96136daf-e54e-4ba9-8ade-23ec9a99924b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:44.144042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96136daf-e54e-4ba9-8ade-23ec9a99924b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:44.144320 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96136daf-e54e-4ba9-8ade-23ec9a99924b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:44.145582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96136daf-e54e-4ba9-8ade-23ec9a99924b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:44.159222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96136daf-e54e-4ba9-8ade-23ec9a99924b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:44.159222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1316/2601] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:49:44.162701 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02c836-2edc-4ff4-b24f-2c957e748e8f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:44.172766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02c836-2edc-4ff4-b24f-2c957e748e8f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:44.179447 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05872d25-573f-4d1f-93f3-f106cfe21c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:44.180190 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05872d25-573f-4d1f-93f3-f106cfe21c01 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:44.180515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05872d25-573f-4d1f-93f3-f106cfe21c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:44.180845 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05872d25-573f-4d1f-93f3-f106cfe21c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:44.193804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05872d25-573f-4d1f-93f3-f106cfe21c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:44.193804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05872d25-573f-4d1f-93f3-f106cfe21c01 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:44.193804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:44.194108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1286/2602] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:49:44.198939 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96136daf-e54e-4ba9-8ade-23ec9a99924b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:44.210149 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96136daf-e54e-4ba9-8ade-23ec9a99924b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:44.214796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89c4a3-5082-43f3-95fb-ed0aa77dd54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:44.227633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89c4a3-5082-43f3-95fb-ed0aa77dd54c admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:44.227633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89c4a3-5082-43f3-95fb-ed0aa77dd54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:44.227633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89c4a3-5082-43f3-95fb-ed0aa77dd54c admin admin] PATH_INFO: `/v3/projects/8632abdaaa904811a43b2e6640f15774/users/baaaa3eaef334031a860864b483e7cab/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:44.231928 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89c4a3-5082-43f3-95fb-ed0aa77dd54c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8632abdaaa904811a43b2e6640f15774, user_id=baaaa3eaef334031a860864b483e7cab, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:44.234617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89c4a3-5082-43f3-95fb-ed0aa77dd54c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:44.319512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1317/2603] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:49:44 2022] PUT /identity/v3/projects/8632abdaaa904811a43b2e6640f15774/users/baaaa3eaef334031a860864b483e7cab/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:44.330159 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069084ed-ffd1-4aec-ae39-0d81bba5b58c None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:44.330450 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069084ed-ffd1-4aec-ae39-0d81bba5b58c None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:44.330774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069084ed-ffd1-4aec-ae39-0d81bba5b58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:44.360754 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-069084ed-ffd1-4aec-ae39-0d81bba5b58c None None] MFA Rules not processed for user `baaaa3eaef334031a860864b483e7cab`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:44.362154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-069084ed-ffd1-4aec-ae39-0d81bba5b58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:44.428852 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1287/2604] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:44.440201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89c4a3-5082-43f3-95fb-ed0aa77dd54c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:44.442599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cb89c4a3-5082-43f3-95fb-ed0aa77dd54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:44.502090 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89c4a3-5082-43f3-95fb-ed0aa77dd54c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:44.507879 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1d1bd-4499-4ea1-9928-7b59ff60d110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:44.508540 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1d1bd-4499-4ea1-9928-7b59ff60d110 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:44.508837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1d1bd-4499-4ea1-9928-7b59ff60d110 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:44.509151 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1d1bd-4499-4ea1-9928-7b59ff60d110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:44.517783 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1d1bd-4499-4ea1-9928-7b59ff60d110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:44.519210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1d1bd-4499-4ea1-9928-7b59ff60d110 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:44.529507 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1318/2605] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:49:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:50.139516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069084ed-ffd1-4aec-ae39-0d81bba5b58c None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:50.140971 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-069084ed-ffd1-4aec-ae39-0d81bba5b58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:50.209093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069084ed-ffd1-4aec-ae39-0d81bba5b58c None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:50.213633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897ec64-886a-4e50-8b9e-4eeb1d5b54c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:50.214564 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897ec64-886a-4e50-8b9e-4eeb1d5b54c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:50.214856 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897ec64-886a-4e50-8b9e-4eeb1d5b54c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:50.215139 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897ec64-886a-4e50-8b9e-4eeb1d5b54c7 admin admin] PATH_INFO: `/v3/users/3452984e01354130828ac96326821503` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:50.216341 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897ec64-886a-4e50-8b9e-4eeb1d5b54c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3452984e01354130828ac96326821503)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:50.218330 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897ec64-886a-4e50-8b9e-4eeb1d5b54c7 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:50.407229 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0897ec64-886a-4e50-8b9e-4eeb1d5b54c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3452984e01354130828ac96326821503'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:50.407892 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0897ec64-886a-4e50-8b9e-4eeb1d5b54c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3452984e01354130828ac96326821503'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:50.408505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0897ec64-886a-4e50-8b9e-4eeb1d5b54c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3452984e01354130828ac96326821503'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:50.416366 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0897ec64-886a-4e50-8b9e-4eeb1d5b54c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3452984e01354130828ac96326821503'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:50.428503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0897ec64-886a-4e50-8b9e-4eeb1d5b54c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3452984e01354130828ac96326821503'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:50.501262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1288/2606] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:49:50 2022] DELETE /identity/v3/users/3452984e01354130828ac96326821503 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:49:50.506485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0340d-376d-486a-8946-4b78fbb3eace None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:50.507002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0340d-376d-486a-8946-4b78fbb3eace None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:50.507123 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0340d-376d-486a-8946-4b78fbb3eace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:50.525010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7ea0340d-376d-486a-8946-4b78fbb3eace None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:50.526257 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7ea0340d-376d-486a-8946-4b78fbb3eace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:50.548056 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:50.589863 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1319/2607] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:50.600079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897ec64-886a-4e50-8b9e-4eeb1d5b54c7 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:50.601625 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0897ec64-886a-4e50-8b9e-4eeb1d5b54c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:50.608453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:50.661751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897ec64-886a-4e50-8b9e-4eeb1d5b54c7 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:50.665750 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e73d65-69c4-4f08-a876-944458171125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:50.666504 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e73d65-69c4-4f08-a876-944458171125 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:50.666796 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e73d65-69c4-4f08-a876-944458171125 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:50.667077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e73d65-69c4-4f08-a876-944458171125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:50.676877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e73d65-69c4-4f08-a876-944458171125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:50.678269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e73d65-69c4-4f08-a876-944458171125 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:50.688199 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1289/2608] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:49:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:51.137564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0340d-376d-486a-8946-4b78fbb3eace None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:51.139030 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7ea0340d-376d-486a-8946-4b78fbb3eace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:51.151714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0340d-376d-486a-8946-4b78fbb3eace None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:51.156453 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f8df0-3288-4c76-8b23-51500e76a5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:51.157512 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f8df0-3288-4c76-8b23-51500e76a5a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:51.157816 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f8df0-3288-4c76-8b23-51500e76a5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:51.164667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f8df0-3288-4c76-8b23-51500e76a5a5 admin admin] PATH_INFO: `/v3/projects/a9e404c07902411090953743248f1551` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:51.164667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f8df0-3288-4c76-8b23-51500e76a5a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9e404c07902411090953743248f1551)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:51.164667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f8df0-3288-4c76-8b23-51500e76a5a5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:51.273370 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a91f8df0-3288-4c76-8b23-51500e76a5a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9e404c07902411090953743248f1551', 'request_id': 'req-a91f8df0-3288-4c76-8b23-51500e76a5a5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:51.279556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a91f8df0-3288-4c76-8b23-51500e76a5a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9e404c07902411090953743248f1551', 'request_id': 'req-a91f8df0-3288-4c76-8b23-51500e76a5a5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:51.387009 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a91f8df0-3288-4c76-8b23-51500e76a5a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9e404c07902411090953743248f1551', 'request_id': 'req-a91f8df0-3288-4c76-8b23-51500e76a5a5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:51.398655 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a91f8df0-3288-4c76-8b23-51500e76a5a5 admin admin] The token cache is being invalidate because project a9e404c07902411090953743248f1551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:49:51.398948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a91f8df0-3288-4c76-8b23-51500e76a5a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:51.399576 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a91f8df0-3288-4c76-8b23-51500e76a5a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:51.401265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1320/2609] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:49:51 2022] DELETE /identity/v3/projects/a9e404c07902411090953743248f1551 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:51.406114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72801b-c9c0-497c-9a2b-08246fd60be3 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:51.406624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72801b-c9c0-497c-9a2b-08246fd60be3 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:51.406723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72801b-c9c0-497c-9a2b-08246fd60be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:51.425318 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-5a72801b-c9c0-497c-9a2b-08246fd60be3 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:51.427158 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5a72801b-c9c0-497c-9a2b-08246fd60be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:51.485178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1290/2610] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:51.493668 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b8868-934e-4fae-b052-c12868efd531 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:51.494000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b8868-934e-4fae-b052-c12868efd531 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:51.494287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b8868-934e-4fae-b052-c12868efd531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:51.512858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-bd9b8868-934e-4fae-b052-c12868efd531 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:51.514100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bd9b8868-934e-4fae-b052-c12868efd531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:51.522109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1321/2611] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:51.526282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72801b-c9c0-497c-9a2b-08246fd60be3 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:51.536749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72801b-c9c0-497c-9a2b-08246fd60be3 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:51.541279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8225c-0597-48d5-aaaf-36dcfef253fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:51.543695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8225c-0597-48d5-aaaf-36dcfef253fb admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:51.543695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8225c-0597-48d5-aaaf-36dcfef253fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:51.543695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8225c-0597-48d5-aaaf-36dcfef253fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:51.543695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8225c-0597-48d5-aaaf-36dcfef253fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:49:51.543695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8225c-0597-48d5-aaaf-36dcfef253fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:51.545285 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8225c-0597-48d5-aaaf-36dcfef253fb admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:51.557869 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1291/2612] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:49:51.561782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b8868-934e-4fae-b052-c12868efd531 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:51.571603 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b8868-934e-4fae-b052-c12868efd531 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:51.576327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade12d9-e9e7-4be3-980c-b332b114a268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:51.576979 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade12d9-e9e7-4be3-980c-b332b114a268 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:51.577300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade12d9-e9e7-4be3-980c-b332b114a268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:51.577584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade12d9-e9e7-4be3-980c-b332b114a268 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:51.578305 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade12d9-e9e7-4be3-980c-b332b114a268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:51.580205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade12d9-e9e7-4be3-980c-b332b114a268 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:51.668680 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1322/2613] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:49:51 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:49:51.672734 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8225c-0597-48d5-aaaf-36dcfef253fb admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:51.682953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8225c-0597-48d5-aaaf-36dcfef253fb admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:51.739397 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbf1d9-0cc6-443d-b2b0-275eafac3ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:51.740065 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbf1d9-0cc6-443d-b2b0-275eafac3ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:51.740350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbf1d9-0cc6-443d-b2b0-275eafac3ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:51.740628 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbf1d9-0cc6-443d-b2b0-275eafac3ac3 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:51.741388 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbf1d9-0cc6-443d-b2b0-275eafac3ac3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:51.743301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbf1d9-0cc6-443d-b2b0-275eafac3ac3 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:51.839566 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1292/2614] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:49:51 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:49:51.843956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade12d9-e9e7-4be3-980c-b332b114a268 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:51.855061 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade12d9-e9e7-4be3-980c-b332b114a268 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:51.859667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca44ac3-4ebf-430e-ae00-87240552dfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:51.860317 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca44ac3-4ebf-430e-ae00-87240552dfb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:51.860602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca44ac3-4ebf-430e-ae00-87240552dfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:51.860882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca44ac3-4ebf-430e-ae00-87240552dfb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:51.861647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca44ac3-4ebf-430e-ae00-87240552dfb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:51.863229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca44ac3-4ebf-430e-ae00-87240552dfb2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:51.873517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1323/2615] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:49:51.877432 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbf1d9-0cc6-443d-b2b0-275eafac3ac3 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:51.888077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbf1d9-0cc6-443d-b2b0-275eafac3ac3 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:51.894011 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c81ac-74d0-4476-9480-a90001f6c511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:51.894669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c81ac-74d0-4476-9480-a90001f6c511 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:51.895479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c81ac-74d0-4476-9480-a90001f6c511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:51.895798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c81ac-74d0-4476-9480-a90001f6c511 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:51.896676 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c81ac-74d0-4476-9480-a90001f6c511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:51.898386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c81ac-74d0-4476-9480-a90001f6c511 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:51.908954 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1293/2616] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:49:51.913718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca44ac3-4ebf-430e-ae00-87240552dfb2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:51.924221 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca44ac3-4ebf-430e-ae00-87240552dfb2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:51.934746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40db744-449a-4d9c-b474-3fa3331e64d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:51.934746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40db744-449a-4d9c-b474-3fa3331e64d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:51.934746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40db744-449a-4d9c-b474-3fa3331e64d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:51.934746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40db744-449a-4d9c-b474-3fa3331e64d3 admin admin] PATH_INFO: `/v3/projects/58c0e4fa5fa246f08bb61703a42e2c59/users/6c14e93cced14b1b95e2de8fedce31e8/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:51.947988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40db744-449a-4d9c-b474-3fa3331e64d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c0e4fa5fa246f08bb61703a42e2c59, user_id=6c14e93cced14b1b95e2de8fedce31e8, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:51.950778 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40db744-449a-4d9c-b474-3fa3331e64d3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:52.043865 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1324/2617] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:49:51 2022] PUT /identity/v3/projects/58c0e4fa5fa246f08bb61703a42e2c59/users/6c14e93cced14b1b95e2de8fedce31e8/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:52.050715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e00a8-8578-42ee-affc-4829770db2f8 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:52.051014 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e00a8-8578-42ee-affc-4829770db2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:52.051555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e00a8-8578-42ee-affc-4829770db2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:52.070435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2a9e00a8-8578-42ee-affc-4829770db2f8 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:52.071723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2a9e00a8-8578-42ee-affc-4829770db2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:52.080052 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:52.134289 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1294/2618] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:52.144730 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40db744-449a-4d9c-b474-3fa3331e64d3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:52.147038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d40db744-449a-4d9c-b474-3fa3331e64d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:52.197169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:52.208575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40db744-449a-4d9c-b474-3fa3331e64d3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:52.212285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864abf21-7e04-42c2-8e80-35b9a5d7bb55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:52.212912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864abf21-7e04-42c2-8e80-35b9a5d7bb55 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:52.213230 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864abf21-7e04-42c2-8e80-35b9a5d7bb55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:52.213513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864abf21-7e04-42c2-8e80-35b9a5d7bb55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:52.222426 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864abf21-7e04-42c2-8e80-35b9a5d7bb55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:52.223973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864abf21-7e04-42c2-8e80-35b9a5d7bb55 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:52.233389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1325/2619] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:49:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:49:53.559337 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db03f4d-ee59-45e9-b607-6586349f8562 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:53.559847 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db03f4d-ee59-45e9-b607-6586349f8562 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:53.559943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db03f4d-ee59-45e9-b607-6586349f8562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:53.596795 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-9db03f4d-ee59-45e9-b607-6586349f8562 None None] MFA Rules not processed for user `6c14e93cced14b1b95e2de8fedce31e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:53.598102 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9db03f4d-ee59-45e9-b607-6586349f8562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:53.669004 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1295/2620] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:53.679701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864abf21-7e04-42c2-8e80-35b9a5d7bb55 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:53.681203 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-864abf21-7e04-42c2-8e80-35b9a5d7bb55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:53.741480 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864abf21-7e04-42c2-8e80-35b9a5d7bb55 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:53.745198 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff45d68-3d93-4164-9d47-736236d5f6b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:53.745977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff45d68-3d93-4164-9d47-736236d5f6b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:53.746262 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff45d68-3d93-4164-9d47-736236d5f6b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:53.746535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff45d68-3d93-4164-9d47-736236d5f6b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:53.755432 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff45d68-3d93-4164-9d47-736236d5f6b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:53.756840 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff45d68-3d93-4164-9d47-736236d5f6b3 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:53.766169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1326/2621] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:49:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:54.486404 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db03f4d-ee59-45e9-b607-6586349f8562 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:54.487835 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9db03f4d-ee59-45e9-b607-6586349f8562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:54.502334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db03f4d-ee59-45e9-b607-6586349f8562 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:54.507587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63118728-6dad-4d3a-b91f-b8ac0856a7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:54.508460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63118728-6dad-4d3a-b91f-b8ac0856a7de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:54.508748 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63118728-6dad-4d3a-b91f-b8ac0856a7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:54.509028 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63118728-6dad-4d3a-b91f-b8ac0856a7de admin admin] PATH_INFO: `/v3/users/baaaa3eaef334031a860864b483e7cab` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:54.510251 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63118728-6dad-4d3a-b91f-b8ac0856a7de admin admin] RBAC: Authorizing `identity:delete_user(user_id=baaaa3eaef334031a860864b483e7cab)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:54.512194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63118728-6dad-4d3a-b91f-b8ac0856a7de admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:54.710311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-63118728-6dad-4d3a-b91f-b8ac0856a7de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baaaa3eaef334031a860864b483e7cab'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:54.710958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-63118728-6dad-4d3a-b91f-b8ac0856a7de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baaaa3eaef334031a860864b483e7cab'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:54.711555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-63118728-6dad-4d3a-b91f-b8ac0856a7de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baaaa3eaef334031a860864b483e7cab'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:54.718373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-63118728-6dad-4d3a-b91f-b8ac0856a7de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baaaa3eaef334031a860864b483e7cab'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:54.731144 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-63118728-6dad-4d3a-b91f-b8ac0856a7de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baaaa3eaef334031a860864b483e7cab'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:54.814782 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1296/2622] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:49:54 2022] DELETE /identity/v3/users/baaaa3eaef334031a860864b483e7cab => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:54.819893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb86a1e-79f9-4374-a910-b3de4f3aebfb None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:54.830339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb86a1e-79f9-4374-a910-b3de4f3aebfb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:54.830339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb86a1e-79f9-4374-a910-b3de4f3aebfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:54.837980 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8cb86a1e-79f9-4374-a910-b3de4f3aebfb None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:54.839180 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8cb86a1e-79f9-4374-a910-b3de4f3aebfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:54.896997 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1327/2623] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:54.906604 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63118728-6dad-4d3a-b91f-b8ac0856a7de admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:54.908010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-63118728-6dad-4d3a-b91f-b8ac0856a7de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:54.932581 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:54.971198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63118728-6dad-4d3a-b91f-b8ac0856a7de admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:54.975736 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13442a-4cf2-4963-bbae-dcd9b5ed2b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:54.976386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13442a-4cf2-4963-bbae-dcd9b5ed2b32 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:54.976682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13442a-4cf2-4963-bbae-dcd9b5ed2b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:54.976966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13442a-4cf2-4963-bbae-dcd9b5ed2b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:54.985902 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13442a-4cf2-4963-bbae-dcd9b5ed2b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:54.987348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13442a-4cf2-4963-bbae-dcd9b5ed2b32 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:54.997257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1297/2624] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:49:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:55.388828 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb86a1e-79f9-4374-a910-b3de4f3aebfb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:55.390292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8cb86a1e-79f9-4374-a910-b3de4f3aebfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:55.402902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb86a1e-79f9-4374-a910-b3de4f3aebfb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:55.407793 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e1a31-7045-46b7-8b57-0612ebc379a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:55.408615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e1a31-7045-46b7-8b57-0612ebc379a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:55.408901 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e1a31-7045-46b7-8b57-0612ebc379a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:55.409222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e1a31-7045-46b7-8b57-0612ebc379a1 admin admin] PATH_INFO: `/v3/projects/8632abdaaa904811a43b2e6640f15774` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:55.411081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e1a31-7045-46b7-8b57-0612ebc379a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8632abdaaa904811a43b2e6640f15774)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:55.413165 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e1a31-7045-46b7-8b57-0612ebc379a1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:55.533010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ba6e1a31-7045-46b7-8b57-0612ebc379a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8632abdaaa904811a43b2e6640f15774', 'request_id': 'req-ba6e1a31-7045-46b7-8b57-0612ebc379a1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:55.533010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ba6e1a31-7045-46b7-8b57-0612ebc379a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8632abdaaa904811a43b2e6640f15774', 'request_id': 'req-ba6e1a31-7045-46b7-8b57-0612ebc379a1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:55.542539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280ca9f-b3cf-4586-975a-c5d6c599c242 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:55.543042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280ca9f-b3cf-4586-975a-c5d6c599c242 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:55.543334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280ca9f-b3cf-4586-975a-c5d6c599c242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:55.561305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-a280ca9f-b3cf-4586-975a-c5d6c599c242 None None] MFA Rules not processed for user `6c14e93cced14b1b95e2de8fedce31e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:55.562477 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a280ca9f-b3cf-4586-975a-c5d6c599c242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:55.603582 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ba6e1a31-7045-46b7-8b57-0612ebc379a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8632abdaaa904811a43b2e6640f15774', 'request_id': 'req-ba6e1a31-7045-46b7-8b57-0612ebc379a1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:55.615926 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ba6e1a31-7045-46b7-8b57-0612ebc379a1 admin admin] The token cache is being invalidate because project 8632abdaaa904811a43b2e6640f15774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:49:55.616227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ba6e1a31-7045-46b7-8b57-0612ebc379a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:55.616861 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ba6e1a31-7045-46b7-8b57-0612ebc379a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:49:55.618561 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1328/2625] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:49:55 2022] DELETE /identity/v3/projects/8632abdaaa904811a43b2e6640f15774 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:55.626480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1298/2626] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:49:55.626656 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a90f4-c1ff-4d9a-945e-0acc8d3d1fdd None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:55.626656 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a90f4-c1ff-4d9a-945e-0acc8d3d1fdd None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:55.626656 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a90f4-c1ff-4d9a-945e-0acc8d3d1fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:55.642876 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-cb2a90f4-c1ff-4d9a-945e-0acc8d3d1fdd None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:55.644094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cb2a90f4-c1ff-4d9a-945e-0acc8d3d1fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:55.653706 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:55.717517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1329/2627] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:49:55.727434 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f23fa1-32f3-48d2-9fbb-dd28a575479e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:55.727925 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f23fa1-32f3-48d2-9fbb-dd28a575479e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:55.728026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f23fa1-32f3-48d2-9fbb-dd28a575479e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:55.747045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b1f23fa1-32f3-48d2-9fbb-dd28a575479e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:55.748258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b1f23fa1-32f3-48d2-9fbb-dd28a575479e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:55.756796 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1299/2628] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:55.766413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a90f4-c1ff-4d9a-945e-0acc8d3d1fdd None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:55.772517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a90f4-c1ff-4d9a-945e-0acc8d3d1fdd None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:55.777276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859ce13-b61b-4013-9b73-873aa69e8638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:55.777958 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859ce13-b61b-4013-9b73-873aa69e8638 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:55.778246 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859ce13-b61b-4013-9b73-873aa69e8638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:55.778526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859ce13-b61b-4013-9b73-873aa69e8638 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:55.779259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859ce13-b61b-4013-9b73-873aa69e8638 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:49:55.779571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859ce13-b61b-4013-9b73-873aa69e8638 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:55.781442 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859ce13-b61b-4013-9b73-873aa69e8638 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:55.795618 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1330/2629] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:49:55.800930 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f23fa1-32f3-48d2-9fbb-dd28a575479e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:55.810647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f23fa1-32f3-48d2-9fbb-dd28a575479e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:55.815706 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538161d6-6e74-42ce-9c35-1fc9a7884eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:55.816438 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538161d6-6e74-42ce-9c35-1fc9a7884eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:55.817556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538161d6-6e74-42ce-9c35-1fc9a7884eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:55.817556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538161d6-6e74-42ce-9c35-1fc9a7884eb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:55.818488 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538161d6-6e74-42ce-9c35-1fc9a7884eb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:55.820546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538161d6-6e74-42ce-9c35-1fc9a7884eb0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:55.915870 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1300/2630] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:49:55 2022] POST /identity/v3/projects => generated 399 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:49:55.920173 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859ce13-b61b-4013-9b73-873aa69e8638 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:55.932509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859ce13-b61b-4013-9b73-873aa69e8638 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:55.990920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b74ee-b752-4bdb-8b5e-465fd97f8a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:55.991593 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b74ee-b752-4bdb-8b5e-465fd97f8a01 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:55.991886 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b74ee-b752-4bdb-8b5e-465fd97f8a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:55.992171 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b74ee-b752-4bdb-8b5e-465fd97f8a01 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:55.992874 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b74ee-b752-4bdb-8b5e-465fd97f8a01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:55.994840 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b74ee-b752-4bdb-8b5e-465fd97f8a01 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:56.110543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1331/2631] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:49:55 2022] POST /identity/v3/users => generated 300 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:49:56.112975 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538161d6-6e74-42ce-9c35-1fc9a7884eb0 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:56.123381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538161d6-6e74-42ce-9c35-1fc9a7884eb0 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:56.127956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf47867-f0e1-4213-bc19-d35a4c2c7b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:56.130105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf47867-f0e1-4213-bc19-d35a4c2c7b92 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:56.130105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf47867-f0e1-4213-bc19-d35a4c2c7b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:56.130105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf47867-f0e1-4213-bc19-d35a4c2c7b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:56.130105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf47867-f0e1-4213-bc19-d35a4c2c7b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:56.131512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf47867-f0e1-4213-bc19-d35a4c2c7b92 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:56.142253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1301/2632] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:49:56.147269 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b74ee-b752-4bdb-8b5e-465fd97f8a01 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:56.157390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b74ee-b752-4bdb-8b5e-465fd97f8a01 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:56.162895 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5f503-d056-4556-ae7d-aa5fc3648b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:56.163559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5f503-d056-4556-ae7d-aa5fc3648b7d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:56.163890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5f503-d056-4556-ae7d-aa5fc3648b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:56.164160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5f503-d056-4556-ae7d-aa5fc3648b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:56.164891 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5f503-d056-4556-ae7d-aa5fc3648b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:56.166513 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5f503-d056-4556-ae7d-aa5fc3648b7d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:56.177719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1332/2633] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:49:56.183528 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf47867-f0e1-4213-bc19-d35a4c2c7b92 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:56.206167 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf47867-f0e1-4213-bc19-d35a4c2c7b92 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:56.212561 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257a1b9-8ff6-41f7-affd-afcdcabeb848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:56.213496 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257a1b9-8ff6-41f7-affd-afcdcabeb848 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:56.213779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257a1b9-8ff6-41f7-affd-afcdcabeb848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:56.214059 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257a1b9-8ff6-41f7-affd-afcdcabeb848 admin admin] PATH_INFO: `/v3/projects/0230b9aa9037415982bd1e39f2407012/users/109293e3bc9148be9bb0ab5e9f0b8b63/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:56.231544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257a1b9-8ff6-41f7-affd-afcdcabeb848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0230b9aa9037415982bd1e39f2407012, user_id=109293e3bc9148be9bb0ab5e9f0b8b63, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:56.234436 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257a1b9-8ff6-41f7-affd-afcdcabeb848 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:56.307558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1302/2634] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:49:56 2022] PUT /identity/v3/projects/0230b9aa9037415982bd1e39f2407012/users/109293e3bc9148be9bb0ab5e9f0b8b63/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:56.317912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f1903-2841-47cd-bb87-9eef310bfa9a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:56.318220 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f1903-2841-47cd-bb87-9eef310bfa9a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:56.318500 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f1903-2841-47cd-bb87-9eef310bfa9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:56.347618 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0d3f1903-2841-47cd-bb87-9eef310bfa9a None None] MFA Rules not processed for user `109293e3bc9148be9bb0ab5e9f0b8b63`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:56.349445 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0d3f1903-2841-47cd-bb87-9eef310bfa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:56.415858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1333/2635] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:56.420386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257a1b9-8ff6-41f7-affd-afcdcabeb848 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:56.485151 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257a1b9-8ff6-41f7-affd-afcdcabeb848 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:56.489883 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f624fb-06e3-48dc-8e8e-1d22b5975beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:56.490539 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f624fb-06e3-48dc-8e8e-1d22b5975beb admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:56.490825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f624fb-06e3-48dc-8e8e-1d22b5975beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:56.491110 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f624fb-06e3-48dc-8e8e-1d22b5975beb admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:56.491803 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f624fb-06e3-48dc-8e8e-1d22b5975beb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:56.493743 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f624fb-06e3-48dc-8e8e-1d22b5975beb admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:56.496957 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:56.639877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1303/2636] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:49:56 2022] POST /identity/v3/projects => generated 401 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:49:56.645400 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f1903-2841-47cd-bb87-9eef310bfa9a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:56.656452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f1903-2841-47cd-bb87-9eef310bfa9a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:56.661209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:56.714339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48384767-1dca-4121-91b4-bd0ccb079dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:56.715018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48384767-1dca-4121-91b4-bd0ccb079dbb admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:56.715306 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48384767-1dca-4121-91b4-bd0ccb079dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:56.715672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48384767-1dca-4121-91b4-bd0ccb079dbb admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:56.716256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48384767-1dca-4121-91b4-bd0ccb079dbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:56.718212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48384767-1dca-4121-91b4-bd0ccb079dbb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:56.804279 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1334/2637] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:49:56 2022] POST /identity/v3/users => generated 307 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:49:56.808153 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f624fb-06e3-48dc-8e8e-1d22b5975beb admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:56.818824 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f624fb-06e3-48dc-8e8e-1d22b5975beb admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:56.823414 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09703bd-7c35-455e-bbbd-75781146bb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:56.824069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09703bd-7c35-455e-bbbd-75781146bb97 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:56.824355 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09703bd-7c35-455e-bbbd-75781146bb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:56.824673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09703bd-7c35-455e-bbbd-75781146bb97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:56.825435 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09703bd-7c35-455e-bbbd-75781146bb97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:56.829350 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09703bd-7c35-455e-bbbd-75781146bb97 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:56.839008 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1304/2638] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:49:56.843153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48384767-1dca-4121-91b4-bd0ccb079dbb admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:56.853848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48384767-1dca-4121-91b4-bd0ccb079dbb admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:56.858405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a80e54-0765-443c-b6c8-597149b7f1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:56.859343 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a80e54-0765-443c-b6c8-597149b7f1e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:56.859632 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a80e54-0765-443c-b6c8-597149b7f1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:56.859917 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a80e54-0765-443c-b6c8-597149b7f1e0 admin admin] PATH_INFO: `/v3/projects/b476c44fa28345bdbae2359d30b920af/users/9046a6e57566497596d7b657719e71d9/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:56.874834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a80e54-0765-443c-b6c8-597149b7f1e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b476c44fa28345bdbae2359d30b920af, user_id=9046a6e57566497596d7b657719e71d9, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:56.877516 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a80e54-0765-443c-b6c8-597149b7f1e0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:56.969810 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1335/2639] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:49:56 2022] PUT /identity/v3/projects/b476c44fa28345bdbae2359d30b920af/users/9046a6e57566497596d7b657719e71d9/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:49:56.973964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09703bd-7c35-455e-bbbd-75781146bb97 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:56.984652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09703bd-7c35-455e-bbbd-75781146bb97 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:57.039153 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6bfb9-2f8e-459a-92ae-8177ed69bfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:57.039820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6bfb9-2f8e-459a-92ae-8177ed69bfc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:57.040104 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6bfb9-2f8e-459a-92ae-8177ed69bfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:57.040381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6bfb9-2f8e-459a-92ae-8177ed69bfc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:57.041121 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6bfb9-2f8e-459a-92ae-8177ed69bfc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:57.042705 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6bfb9-2f8e-459a-92ae-8177ed69bfc7 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:57.053278 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1305/2640] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:49:57.057639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a80e54-0765-443c-b6c8-597149b7f1e0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:57.068524 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a80e54-0765-443c-b6c8-597149b7f1e0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:57.073303 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f082d53-9486-4c12-b366-80b56e89a1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:57.074179 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082d53-9486-4c12-b366-80b56e89a1b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:57.074467 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082d53-9486-4c12-b366-80b56e89a1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:57.074744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082d53-9486-4c12-b366-80b56e89a1b2 admin admin] PATH_INFO: `/v3/projects/b476c44fa28345bdbae2359d30b920af/users/9046a6e57566497596d7b657719e71d9/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:57.076682 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f082d53-9486-4c12-b366-80b56e89a1b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b476c44fa28345bdbae2359d30b920af, user_id=9046a6e57566497596d7b657719e71d9, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:57.079359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f082d53-9486-4c12-b366-80b56e89a1b2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:57.089726 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1336/2641] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:49:57 2022] PUT /identity/v3/projects/b476c44fa28345bdbae2359d30b920af/users/9046a6e57566497596d7b657719e71d9/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:49:57.101409 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfc564-f85f-4f6d-9516-892f16ce7a1e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:57.105359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfc564-f85f-4f6d-9516-892f16ce7a1e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:57.105359 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfc564-f85f-4f6d-9516-892f16ce7a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:57.128021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b3dfc564-f85f-4f6d-9516-892f16ce7a1e None None] MFA Rules not processed for user `9046a6e57566497596d7b657719e71d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:57.129935 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b3dfc564-f85f-4f6d-9516-892f16ce7a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:57.198111 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1306/2642] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:49:57 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:57.208908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f082d53-9486-4c12-b366-80b56e89a1b2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:57.210657 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6f082d53-9486-4c12-b366-80b56e89a1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:57.269392 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f082d53-9486-4c12-b366-80b56e89a1b2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:57.274390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc655ee-3311-4bac-a951-316e86e3cf0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:57.275065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc655ee-3311-4bac-a951-316e86e3cf0a service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:57.275386 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc655ee-3311-4bac-a951-316e86e3cf0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:57.275671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc655ee-3311-4bac-a951-316e86e3cf0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:57.285956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc655ee-3311-4bac-a951-316e86e3cf0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:57.287599 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc655ee-3311-4bac-a951-316e86e3cf0a service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:57.297727 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1337/2643] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:49:57 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:57.500444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfc564-f85f-4f6d-9516-892f16ce7a1e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:57.510640 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfc564-f85f-4f6d-9516-892f16ce7a1e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:57.515309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d25b9-e1a9-4cdc-a4bc-d57174a07b69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:57.515309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d25b9-e1a9-4cdc-a4bc-d57174a07b69 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:57.515576 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d25b9-e1a9-4cdc-a4bc-d57174a07b69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:57.515703 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d25b9-e1a9-4cdc-a4bc-d57174a07b69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:57.571210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:57.574913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d25b9-e1a9-4cdc-a4bc-d57174a07b69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:57.576349 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d25b9-e1a9-4cdc-a4bc-d57174a07b69 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:57.585760 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1307/2644] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:49:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:50:02.566383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc655ee-3311-4bac-a951-316e86e3cf0a service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:02.631277 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc655ee-3311-4bac-a951-316e86e3cf0a service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:02.635793 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070734c-ec78-484c-a93d-71d5b2828fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:02.636664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070734c-ec78-484c-a93d-71d5b2828fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:02.636946 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070734c-ec78-484c-a93d-71d5b2828fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:02.637256 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070734c-ec78-484c-a93d-71d5b2828fc0 admin admin] PATH_INFO: `/v3/users/109293e3bc9148be9bb0ab5e9f0b8b63` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:02.638418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070734c-ec78-484c-a93d-71d5b2828fc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=109293e3bc9148be9bb0ab5e9f0b8b63)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:02.640412 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070734c-ec78-484c-a93d-71d5b2828fc0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:02.645841 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:02.789152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5070734c-ec78-484c-a93d-71d5b2828fc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '109293e3bc9148be9bb0ab5e9f0b8b63'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:02.801959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5070734c-ec78-484c-a93d-71d5b2828fc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '109293e3bc9148be9bb0ab5e9f0b8b63'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:02.871840 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5070734c-ec78-484c-a93d-71d5b2828fc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '109293e3bc9148be9bb0ab5e9f0b8b63'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:02.872457 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5070734c-ec78-484c-a93d-71d5b2828fc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '109293e3bc9148be9bb0ab5e9f0b8b63'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:02.879762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5070734c-ec78-484c-a93d-71d5b2828fc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '109293e3bc9148be9bb0ab5e9f0b8b63'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:02.885788 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1338/2645] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:50:02 2022] DELETE /identity/v3/users/109293e3bc9148be9bb0ab5e9f0b8b63 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:50:02.890476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef4301-6bd1-458d-a17a-5ae642edca53 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:02.890971 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef4301-6bd1-458d-a17a-5ae642edca53 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:02.891082 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef4301-6bd1-458d-a17a-5ae642edca53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:02.909024 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b0ef4301-6bd1-458d-a17a-5ae642edca53 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:02.910927 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b0ef4301-6bd1-458d-a17a-5ae642edca53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:02.967775 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1308/2646] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:50:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:02.978982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070734c-ec78-484c-a93d-71d5b2828fc0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:02.980400 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5070734c-ec78-484c-a93d-71d5b2828fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:03.039398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070734c-ec78-484c-a93d-71d5b2828fc0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:03.043071 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101452a6-7bc5-40bd-99d5-dfbf37c8f328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:03.043723 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101452a6-7bc5-40bd-99d5-dfbf37c8f328 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:03.044012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101452a6-7bc5-40bd-99d5-dfbf37c8f328 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:03.044326 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101452a6-7bc5-40bd-99d5-dfbf37c8f328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:03.054673 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101452a6-7bc5-40bd-99d5-dfbf37c8f328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:03.056041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101452a6-7bc5-40bd-99d5-dfbf37c8f328 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:03.065403 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1339/2647] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:50:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:50:03.525747 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef4301-6bd1-458d-a17a-5ae642edca53 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:03.527316 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b0ef4301-6bd1-458d-a17a-5ae642edca53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:03.540722 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef4301-6bd1-458d-a17a-5ae642edca53 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:03.546051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc19a91-8ada-44c0-a62e-195f1a433f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:03.550390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc19a91-8ada-44c0-a62e-195f1a433f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:03.550390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc19a91-8ada-44c0-a62e-195f1a433f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:03.550390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc19a91-8ada-44c0-a62e-195f1a433f5c admin admin] PATH_INFO: `/v3/projects/0230b9aa9037415982bd1e39f2407012` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:03.550390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc19a91-8ada-44c0-a62e-195f1a433f5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0230b9aa9037415982bd1e39f2407012)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:03.551302 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc19a91-8ada-44c0-a62e-195f1a433f5c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:03.684254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6cc19a91-8ada-44c0-a62e-195f1a433f5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0230b9aa9037415982bd1e39f2407012', 'request_id': 'req-6cc19a91-8ada-44c0-a62e-195f1a433f5c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:03.689639 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6cc19a91-8ada-44c0-a62e-195f1a433f5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0230b9aa9037415982bd1e39f2407012', 'request_id': 'req-6cc19a91-8ada-44c0-a62e-195f1a433f5c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:03.696564 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6cc19a91-8ada-44c0-a62e-195f1a433f5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0230b9aa9037415982bd1e39f2407012', 'request_id': 'req-6cc19a91-8ada-44c0-a62e-195f1a433f5c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:03.768691 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6cc19a91-8ada-44c0-a62e-195f1a433f5c admin admin] The token cache is being invalidate because project 0230b9aa9037415982bd1e39f2407012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:50:03.768983 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6cc19a91-8ada-44c0-a62e-195f1a433f5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:03.769682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6cc19a91-8ada-44c0-a62e-195f1a433f5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:03.771269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1309/2648] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:50:03 2022] DELETE /identity/v3/projects/0230b9aa9037415982bd1e39f2407012 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:50:03.775259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101452a6-7bc5-40bd-99d5-dfbf37c8f328 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:03.784879 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101452a6-7bc5-40bd-99d5-dfbf37c8f328 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:03.841948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242e962-ee09-47b3-b53f-9aebcbb8cd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:03.842822 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242e962-ee09-47b3-b53f-9aebcbb8cd17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:03.843114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242e962-ee09-47b3-b53f-9aebcbb8cd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:03.843395 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242e962-ee09-47b3-b53f-9aebcbb8cd17 admin admin] PATH_INFO: `/v3/users/9046a6e57566497596d7b657719e71d9` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:03.844579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242e962-ee09-47b3-b53f-9aebcbb8cd17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9046a6e57566497596d7b657719e71d9)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:03.846520 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242e962-ee09-47b3-b53f-9aebcbb8cd17 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:03.931397 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:04.040852 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6242e962-ee09-47b3-b53f-9aebcbb8cd17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9046a6e57566497596d7b657719e71d9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:04.053440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6242e962-ee09-47b3-b53f-9aebcbb8cd17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9046a6e57566497596d7b657719e71d9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:04.152420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6242e962-ee09-47b3-b53f-9aebcbb8cd17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9046a6e57566497596d7b657719e71d9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:04.153041 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6242e962-ee09-47b3-b53f-9aebcbb8cd17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9046a6e57566497596d7b657719e71d9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:04.161454 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6242e962-ee09-47b3-b53f-9aebcbb8cd17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9046a6e57566497596d7b657719e71d9'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:04.167651 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1340/2649] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:50:03 2022] DELETE /identity/v3/users/9046a6e57566497596d7b657719e71d9 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:50:04.664120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc19a91-8ada-44c0-a62e-195f1a433f5c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:04.666087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6cc19a91-8ada-44c0-a62e-195f1a433f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:04.687944 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc19a91-8ada-44c0-a62e-195f1a433f5c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:04.758948 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961aaf49-e782-40be-9f36-1cccb6a853e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:04.759855 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aaf49-e782-40be-9f36-1cccb6a853e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:04.760152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aaf49-e782-40be-9f36-1cccb6a853e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:04.760437 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aaf49-e782-40be-9f36-1cccb6a853e0 admin admin] PATH_INFO: `/v3/projects/b476c44fa28345bdbae2359d30b920af` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:04.761671 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961aaf49-e782-40be-9f36-1cccb6a853e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b476c44fa28345bdbae2359d30b920af)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:04.763621 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961aaf49-e782-40be-9f36-1cccb6a853e0 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:04.786368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:04.900022 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-961aaf49-e782-40be-9f36-1cccb6a853e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b476c44fa28345bdbae2359d30b920af', 'request_id': 'req-961aaf49-e782-40be-9f36-1cccb6a853e0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:04.907115 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-961aaf49-e782-40be-9f36-1cccb6a853e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b476c44fa28345bdbae2359d30b920af', 'request_id': 'req-961aaf49-e782-40be-9f36-1cccb6a853e0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:04.914455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-961aaf49-e782-40be-9f36-1cccb6a853e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b476c44fa28345bdbae2359d30b920af', 'request_id': 'req-961aaf49-e782-40be-9f36-1cccb6a853e0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:04.982032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-961aaf49-e782-40be-9f36-1cccb6a853e0 admin admin] The token cache is being invalidate because project b476c44fa28345bdbae2359d30b920af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:50:04.982390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-961aaf49-e782-40be-9f36-1cccb6a853e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:04.983149 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-961aaf49-e782-40be-9f36-1cccb6a853e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:04.985894 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1310/2650] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:50:04 2022] DELETE /identity/v3/projects/b476c44fa28345bdbae2359d30b920af => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:50:04.992158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cccdd-720e-4ed1-a312-a45f86a03502 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:04.992782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cccdd-720e-4ed1-a312-a45f86a03502 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:04.992782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cccdd-720e-4ed1-a312-a45f86a03502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:05.012871 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9c8cccdd-720e-4ed1-a312-a45f86a03502 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:05.014110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9c8cccdd-720e-4ed1-a312-a45f86a03502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:05.080161 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1341/2651] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:05.090663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b85f2-6e87-4ed0-9acc-29b10efcf6ed None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:05.090953 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b85f2-6e87-4ed0-9acc-29b10efcf6ed None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:05.091230 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b85f2-6e87-4ed0-9acc-29b10efcf6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:05.110814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-3e4b85f2-6e87-4ed0-9acc-29b10efcf6ed None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:05.112036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3e4b85f2-6e87-4ed0-9acc-29b10efcf6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:05.121257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1311/2652] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:05.125478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cccdd-720e-4ed1-a312-a45f86a03502 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:05.135804 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cccdd-720e-4ed1-a312-a45f86a03502 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:05.140410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd36404-ddd1-401a-a961-b21cc2c33647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:05.141088 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd36404-ddd1-401a-a961-b21cc2c33647 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:05.141390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd36404-ddd1-401a-a961-b21cc2c33647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:05.141705 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd36404-ddd1-401a-a961-b21cc2c33647 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:05.142398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd36404-ddd1-401a-a961-b21cc2c33647 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:50:05.142716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd36404-ddd1-401a-a961-b21cc2c33647 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:05.144306 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd36404-ddd1-401a-a961-b21cc2c33647 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:05.156753 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1342/2653] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:50:05.163583 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b85f2-6e87-4ed0-9acc-29b10efcf6ed None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:05.171168 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b85f2-6e87-4ed0-9acc-29b10efcf6ed None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:05.176321 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fa229-a80d-4243-bf25-2c11d96b989d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:05.176978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fa229-a80d-4243-bf25-2c11d96b989d admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:05.177303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fa229-a80d-4243-bf25-2c11d96b989d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:05.177587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fa229-a80d-4243-bf25-2c11d96b989d admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:05.178275 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fa229-a80d-4243-bf25-2c11d96b989d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:05.180182 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fa229-a80d-4243-bf25-2c11d96b989d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:05.357749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1312/2654] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:50:05 2022] POST /identity/v3/projects => generated 409 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:50:05.359037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd36404-ddd1-401a-a961-b21cc2c33647 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:05.369978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd36404-ddd1-401a-a961-b21cc2c33647 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:05.429899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f45f30-3443-4f9c-8a2c-f3f8f53f3e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:05.430571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f45f30-3443-4f9c-8a2c-f3f8f53f3e9c admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:05.430858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f45f30-3443-4f9c-8a2c-f3f8f53f3e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:05.431145 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f45f30-3443-4f9c-8a2c-f3f8f53f3e9c admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:05.431836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f45f30-3443-4f9c-8a2c-f3f8f53f3e9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:05.433837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f45f30-3443-4f9c-8a2c-f3f8f53f3e9c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:05.547556 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1343/2655] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:50:05 2022] POST /identity/v3/users => generated 305 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:50:05.551834 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fa229-a80d-4243-bf25-2c11d96b989d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:05.563160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fa229-a80d-4243-bf25-2c11d96b989d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:05.568898 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73732d-d0ef-4ce9-a0a2-71da54b7750f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:05.569627 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73732d-d0ef-4ce9-a0a2-71da54b7750f admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:05.569921 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73732d-d0ef-4ce9-a0a2-71da54b7750f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:05.570206 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73732d-d0ef-4ce9-a0a2-71da54b7750f admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:05.570960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73732d-d0ef-4ce9-a0a2-71da54b7750f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:05.572584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73732d-d0ef-4ce9-a0a2-71da54b7750f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:05.588682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1313/2656] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:50:05.590428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f45f30-3443-4f9c-8a2c-f3f8f53f3e9c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:05.603170 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f45f30-3443-4f9c-8a2c-f3f8f53f3e9c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:05.608317 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405c771-f2c9-440e-9440-042431ffcd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:05.609182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405c771-f2c9-440e-9440-042431ffcd5d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:05.609428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405c771-f2c9-440e-9440-042431ffcd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:05.609711 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405c771-f2c9-440e-9440-042431ffcd5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:05.610449 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405c771-f2c9-440e-9440-042431ffcd5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:05.612511 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405c771-f2c9-440e-9440-042431ffcd5d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:05.627280 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1344/2657] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:50:05.628573 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73732d-d0ef-4ce9-a0a2-71da54b7750f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:05.640567 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73732d-d0ef-4ce9-a0a2-71da54b7750f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:05.647334 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587ab4f-61e3-412f-a2a9-43ce636b4b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:05.648297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587ab4f-61e3-412f-a2a9-43ce636b4b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:05.648703 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587ab4f-61e3-412f-a2a9-43ce636b4b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:05.649136 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587ab4f-61e3-412f-a2a9-43ce636b4b00 admin admin] PATH_INFO: `/v3/projects/611a6747c5b64e40bfd6040044e66d93/users/18cf13cee3134a8991bb74b7d831df47/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:05.667051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587ab4f-61e3-412f-a2a9-43ce636b4b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=611a6747c5b64e40bfd6040044e66d93, user_id=18cf13cee3134a8991bb74b7d831df47, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:05.669987 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587ab4f-61e3-412f-a2a9-43ce636b4b00 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:05.753808 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1314/2658] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:50:05 2022] PUT /identity/v3/projects/611a6747c5b64e40bfd6040044e66d93/users/18cf13cee3134a8991bb74b7d831df47/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:50:05.764704 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6f6bd-0991-4d75-96c8-326f541aa133 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:05.765398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6f6bd-0991-4d75-96c8-326f541aa133 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:05.765969 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6f6bd-0991-4d75-96c8-326f541aa133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:05.794549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:05.803858 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f2a6f6bd-0991-4d75-96c8-326f541aa133 None None] MFA Rules not processed for user `18cf13cee3134a8991bb74b7d831df47`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:05.805200 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f2a6f6bd-0991-4d75-96c8-326f541aa133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:05.877442 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1345/2659] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:50:05 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:05.882205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587ab4f-61e3-412f-a2a9-43ce636b4b00 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:05.947817 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587ab4f-61e3-412f-a2a9-43ce636b4b00 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:05.951095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6f6bd-0991-4d75-96c8-326f541aa133 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:05.953601 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0a03b-fed6-4128-ba83-a4646d1c346d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:05.953860 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f2a6f6bd-0991-4d75-96c8-326f541aa133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:05.954353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0a03b-fed6-4128-ba83-a4646d1c346d admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:05.954649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0a03b-fed6-4128-ba83-a4646d1c346d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:05.954933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0a03b-fed6-4128-ba83-a4646d1c346d admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:05.955682 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0a03b-fed6-4128-ba83-a4646d1c346d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:05.958381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0a03b-fed6-4128-ba83-a4646d1c346d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:06.030238 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6f6bd-0991-4d75-96c8-326f541aa133 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:06.037278 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1315/2660] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:50:05 2022] POST /identity/v3/projects => generated 407 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:50:06.037686 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d5c0d-30af-49f3-9668-1d3098545c3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fbaeb7583a456d81cf79f6b101fb1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:06.038347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d5c0d-30af-49f3-9668-1d3098545c3c service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:06.038635 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d5c0d-30af-49f3-9668-1d3098545c3c service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:06.038915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d5c0d-30af-49f3-9668-1d3098545c3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:06.040937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3d6d5c0d-30af-49f3-9668-1d3098545c3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:06.041620 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0a03b-fed6-4128-ba83-a4646d1c346d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:06.054463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d5c0d-30af-49f3-9668-1d3098545c3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:06.057907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d5c0d-30af-49f3-9668-1d3098545c3c service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:06.070838 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1346/2661] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:50:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:50:06.100859 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:06.139888 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0a03b-fed6-4128-ba83-a4646d1c346d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:06.148694 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd27840-9006-444a-a869-d654ca7e44e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:06.149926 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd27840-9006-444a-a869-d654ca7e44e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:06.150375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd27840-9006-444a-a869-d654ca7e44e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:06.150933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd27840-9006-444a-a869-d654ca7e44e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:06.152622 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd27840-9006-444a-a869-d654ca7e44e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:06.156226 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd27840-9006-444a-a869-d654ca7e44e1 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:06.246946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1316/2662] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:50:06 2022] POST /identity/v3/users => generated 310 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:50:06.255371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d5c0d-30af-49f3-9668-1d3098545c3c service placement] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:06.267107 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d5c0d-30af-49f3-9668-1d3098545c3c service placement] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:06.271986 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a404e-2022-4520-ae9b-49a5532654cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:06.272742 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a404e-2022-4520-ae9b-49a5532654cf admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:06.273130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a404e-2022-4520-ae9b-49a5532654cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:06.273499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a404e-2022-4520-ae9b-49a5532654cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:06.274264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a404e-2022-4520-ae9b-49a5532654cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:06.275910 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a404e-2022-4520-ae9b-49a5532654cf admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:06.291575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1347/2663] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:50:06.293169 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd27840-9006-444a-a869-d654ca7e44e1 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:06.305393 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd27840-9006-444a-a869-d654ca7e44e1 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:06.313367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f56c9-dc09-4351-98ec-19bb835d690a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:06.314538 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f56c9-dc09-4351-98ec-19bb835d690a admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:06.315956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f56c9-dc09-4351-98ec-19bb835d690a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:06.316347 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f56c9-dc09-4351-98ec-19bb835d690a admin admin] PATH_INFO: `/v3/projects/a9fcf2dfafab4113881d38748b0824cb/users/0a1c40a3bbc6455bae73ad8a009ea810/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:06.335889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f56c9-dc09-4351-98ec-19bb835d690a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9fcf2dfafab4113881d38748b0824cb, user_id=0a1c40a3bbc6455bae73ad8a009ea810, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:06.338699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f56c9-dc09-4351-98ec-19bb835d690a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:06.412615 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1317/2664] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:50:06 2022] PUT /identity/v3/projects/a9fcf2dfafab4113881d38748b0824cb/users/0a1c40a3bbc6455bae73ad8a009ea810/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:50:06.444391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a404e-2022-4520-ae9b-49a5532654cf admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:06.454806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a404e-2022-4520-ae9b-49a5532654cf admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:06.534005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61614283-d43b-4a8a-83de-71d5e8613383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:06.534698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61614283-d43b-4a8a-83de-71d5e8613383 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:06.534990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61614283-d43b-4a8a-83de-71d5e8613383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:06.535276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61614283-d43b-4a8a-83de-71d5e8613383 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:06.536013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61614283-d43b-4a8a-83de-71d5e8613383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:06.537770 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61614283-d43b-4a8a-83de-71d5e8613383 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:06.548579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1348/2665] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:50:06.553070 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f56c9-dc09-4351-98ec-19bb835d690a admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:06.563647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f56c9-dc09-4351-98ec-19bb835d690a admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:06.569163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2cb7e-09d3-4efa-9137-209e2d9cdef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:06.570099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2cb7e-09d3-4efa-9137-209e2d9cdef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:06.570387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2cb7e-09d3-4efa-9137-209e2d9cdef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:06.570665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2cb7e-09d3-4efa-9137-209e2d9cdef7 admin admin] PATH_INFO: `/v3/projects/a9fcf2dfafab4113881d38748b0824cb/users/0a1c40a3bbc6455bae73ad8a009ea810/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:06.572633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2cb7e-09d3-4efa-9137-209e2d9cdef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9fcf2dfafab4113881d38748b0824cb, user_id=0a1c40a3bbc6455bae73ad8a009ea810, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:06.575365 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2cb7e-09d3-4efa-9137-209e2d9cdef7 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:06.587131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1318/2666] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:50:06 2022] PUT /identity/v3/projects/a9fcf2dfafab4113881d38748b0824cb/users/0a1c40a3bbc6455bae73ad8a009ea810/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:50:06.598745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c1d3f-9751-4243-9ff9-cd16957160e4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:06.599127 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c1d3f-9751-4243-9ff9-cd16957160e4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:06.599481 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c1d3f-9751-4243-9ff9-cd16957160e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:06.626848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-422c1d3f-9751-4243-9ff9-cd16957160e4 None None] MFA Rules not processed for user `0a1c40a3bbc6455bae73ad8a009ea810`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:06.628088 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-422c1d3f-9751-4243-9ff9-cd16957160e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:06.632777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:06.705807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1349/2667] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:50:06 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:50:06.716335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2cb7e-09d3-4efa-9137-209e2d9cdef7 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:06.718336 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b7e2cb7e-09d3-4efa-9137-209e2d9cdef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:06.789685 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2cb7e-09d3-4efa-9137-209e2d9cdef7 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:06.795044 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f34bc3-93b9-43fb-8a70-ca9be4af167d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:06.795731 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f34bc3-93b9-43fb-8a70-ca9be4af167d service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:06.796021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f34bc3-93b9-43fb-8a70-ca9be4af167d service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:06.796354 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f34bc3-93b9-43fb-8a70-ca9be4af167d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:06.810740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f34bc3-93b9-43fb-8a70-ca9be4af167d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:06.812207 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f34bc3-93b9-43fb-8a70-ca9be4af167d service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:06.825105 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1319/2668] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:50:06 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:50:07.116706 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c1d3f-9751-4243-9ff9-cd16957160e4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:07.126282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c1d3f-9751-4243-9ff9-cd16957160e4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:07.130468 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fc04c-caac-4284-a048-0c715b311769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:07.131113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fc04c-caac-4284-a048-0c715b311769 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:07.131401 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fc04c-caac-4284-a048-0c715b311769 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:07.131677 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fc04c-caac-4284-a048-0c715b311769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:07.189251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fc04c-caac-4284-a048-0c715b311769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:07.190692 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fc04c-caac-4284-a048-0c715b311769 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:07.199837 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1350/2669] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:50:07 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:50:10.765213 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4524767-b69e-4644-97bc-29d2c4df5320 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:10.765736 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4524767-b69e-4644-97bc-29d2c4df5320 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:10.765931 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4524767-b69e-4644-97bc-29d2c4df5320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:10.783563 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-c4524767-b69e-4644-97bc-29d2c4df5320 None None] MFA Rules not processed for user `7a5fc191f3464ffe83d5fde2419cb8b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:10.784761 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c4524767-b69e-4644-97bc-29d2c4df5320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:10.840698 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1320/2670] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:50:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:17.196400 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=55 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:17.406244 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fc04c-caac-4284-a048-0c715b311769 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:17.475784 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fc04c-caac-4284-a048-0c715b311769 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:17.481286 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b51790-a57b-48b0-8b54-7e0c188629b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:17.482183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b51790-a57b-48b0-8b54-7e0c188629b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:17.482472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b51790-a57b-48b0-8b54-7e0c188629b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:17.482757 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b51790-a57b-48b0-8b54-7e0c188629b8 admin admin] PATH_INFO: `/v3/users/18cf13cee3134a8991bb74b7d831df47` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:17.483963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b51790-a57b-48b0-8b54-7e0c188629b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18cf13cee3134a8991bb74b7d831df47)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:17.485957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b51790-a57b-48b0-8b54-7e0c188629b8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:17.688071 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-30b51790-a57b-48b0-8b54-7e0c188629b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18cf13cee3134a8991bb74b7d831df47'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:17.701641 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-30b51790-a57b-48b0-8b54-7e0c188629b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18cf13cee3134a8991bb74b7d831df47'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:17.743751 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4524767-b69e-4644-97bc-29d2c4df5320 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:17.745274 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c4524767-b69e-4644-97bc-29d2c4df5320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:17.757295 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4524767-b69e-4644-97bc-29d2c4df5320 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:17.761771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da6967-e9c4-459c-92ad-c01f76425e75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:17.762597 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da6967-e9c4-459c-92ad-c01f76425e75 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:17.762889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da6967-e9c4-459c-92ad-c01f76425e75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:17.763172 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da6967-e9c4-459c-92ad-c01f76425e75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:17.766466 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-95da6967-e9c4-459c-92ad-c01f76425e75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:17.796238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:17.798001 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-30b51790-a57b-48b0-8b54-7e0c188629b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18cf13cee3134a8991bb74b7d831df47'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:17.799056 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-30b51790-a57b-48b0-8b54-7e0c188629b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18cf13cee3134a8991bb74b7d831df47'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:17.812817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-30b51790-a57b-48b0-8b54-7e0c188629b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18cf13cee3134a8991bb74b7d831df47'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:17.819368 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1351/2671] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:50:17 2022] DELETE /identity/v3/users/18cf13cee3134a8991bb74b7d831df47 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:50:17.823976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefac35-c9ec-4b9d-836c-addc03d199b6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:17.824278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefac35-c9ec-4b9d-836c-addc03d199b6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:17.824568 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefac35-c9ec-4b9d-836c-addc03d199b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:17.831462 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da6967-e9c4-459c-92ad-c01f76425e75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:17.833153 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da6967-e9c4-459c-92ad-c01f76425e75 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:17.844335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1321/2672] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:50:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:50:17.846154 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-dcefac35-c9ec-4b9d-836c-addc03d199b6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:17.847452 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dcefac35-c9ec-4b9d-836c-addc03d199b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:17.910345 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1352/2673] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:50:17.923948 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da6967-e9c4-459c-92ad-c01f76425e75 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:17.925679 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-95da6967-e9c4-459c-92ad-c01f76425e75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:17.982971 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da6967-e9c4-459c-92ad-c01f76425e75 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:18.043142 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da5049-7351-483b-b3dd-66e233605392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:18.043804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da5049-7351-483b-b3dd-66e233605392 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:18.044093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da5049-7351-483b-b3dd-66e233605392 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:18.044376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da5049-7351-483b-b3dd-66e233605392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:18.054346 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da5049-7351-483b-b3dd-66e233605392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:18.056316 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da5049-7351-483b-b3dd-66e233605392 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:18.065883 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1322/2674] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:50:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:50:18.508985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefac35-c9ec-4b9d-836c-addc03d199b6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:18.510789 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dcefac35-c9ec-4b9d-836c-addc03d199b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:18.534997 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefac35-c9ec-4b9d-836c-addc03d199b6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:18.540450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518c606-9208-4d20-982c-7c6d183883de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:18.541356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518c606-9208-4d20-982c-7c6d183883de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:18.542037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518c606-9208-4d20-982c-7c6d183883de admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:18.542349 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518c606-9208-4d20-982c-7c6d183883de admin admin] PATH_INFO: `/v3/projects/611a6747c5b64e40bfd6040044e66d93` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:18.546525 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518c606-9208-4d20-982c-7c6d183883de admin admin] RBAC: Authorizing `identity:delete_project(project_id=611a6747c5b64e40bfd6040044e66d93)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:18.549213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518c606-9208-4d20-982c-7c6d183883de admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:18.567130 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:18.648065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c518c606-9208-4d20-982c-7c6d183883de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '611a6747c5b64e40bfd6040044e66d93', 'request_id': 'req-c518c606-9208-4d20-982c-7c6d183883de'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:18.655511 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c518c606-9208-4d20-982c-7c6d183883de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '611a6747c5b64e40bfd6040044e66d93', 'request_id': 'req-c518c606-9208-4d20-982c-7c6d183883de'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:18.712019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c518c606-9208-4d20-982c-7c6d183883de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '611a6747c5b64e40bfd6040044e66d93', 'request_id': 'req-c518c606-9208-4d20-982c-7c6d183883de'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:18.722917 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c518c606-9208-4d20-982c-7c6d183883de admin admin] The token cache is being invalidate because project 611a6747c5b64e40bfd6040044e66d93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:50:18.723228 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c518c606-9208-4d20-982c-7c6d183883de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:18.738642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c518c606-9208-4d20-982c-7c6d183883de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:18.738642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1353/2675] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:50:18 2022] DELETE /identity/v3/projects/611a6747c5b64e40bfd6040044e66d93 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:50:18.741503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da5049-7351-483b-b3dd-66e233605392 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:18.755081 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da5049-7351-483b-b3dd-66e233605392 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:18.817837 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebff860-df10-4306-a0ff-56fc74889036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:18.818988 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebff860-df10-4306-a0ff-56fc74889036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:18.819178 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebff860-df10-4306-a0ff-56fc74889036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:18.819463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebff860-df10-4306-a0ff-56fc74889036 admin admin] PATH_INFO: `/v3/users/0a1c40a3bbc6455bae73ad8a009ea810` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:18.821413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebff860-df10-4306-a0ff-56fc74889036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a1c40a3bbc6455bae73ad8a009ea810)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:18.823384 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebff860-df10-4306-a0ff-56fc74889036 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:18.926014 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:19.052969 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2ebff860-df10-4306-a0ff-56fc74889036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a1c40a3bbc6455bae73ad8a009ea810'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:19.053692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2ebff860-df10-4306-a0ff-56fc74889036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a1c40a3bbc6455bae73ad8a009ea810'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:19.054519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2ebff860-df10-4306-a0ff-56fc74889036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a1c40a3bbc6455bae73ad8a009ea810'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:19.062389 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2ebff860-df10-4306-a0ff-56fc74889036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a1c40a3bbc6455bae73ad8a009ea810'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:19.074947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-2ebff860-df10-4306-a0ff-56fc74889036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a1c40a3bbc6455bae73ad8a009ea810'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:19.144674 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1323/2676] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:50:18 2022] DELETE /identity/v3/users/0a1c40a3bbc6455bae73ad8a009ea810 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:50:19.536835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518c606-9208-4d20-982c-7c6d183883de admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:19.538762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c518c606-9208-4d20-982c-7c6d183883de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:19.555128 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518c606-9208-4d20-982c-7c6d183883de admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:19.629942 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca4a75-dd97-4291-a119-5f21033abc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:19.630793 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca4a75-dd97-4291-a119-5f21033abc2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:19.631082 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca4a75-dd97-4291-a119-5f21033abc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:19.631361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca4a75-dd97-4291-a119-5f21033abc2d admin admin] PATH_INFO: `/v3/projects/a9fcf2dfafab4113881d38748b0824cb` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:19.634615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca4a75-dd97-4291-a119-5f21033abc2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9fcf2dfafab4113881d38748b0824cb)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:19.636530 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca4a75-dd97-4291-a119-5f21033abc2d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:19.749289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c5ca4a75-dd97-4291-a119-5f21033abc2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9fcf2dfafab4113881d38748b0824cb', 'request_id': 'req-c5ca4a75-dd97-4291-a119-5f21033abc2d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:19.756395 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c5ca4a75-dd97-4291-a119-5f21033abc2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9fcf2dfafab4113881d38748b0824cb', 'request_id': 'req-c5ca4a75-dd97-4291-a119-5f21033abc2d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:19.827451 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c5ca4a75-dd97-4291-a119-5f21033abc2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9fcf2dfafab4113881d38748b0824cb', 'request_id': 'req-c5ca4a75-dd97-4291-a119-5f21033abc2d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:19.844667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c5ca4a75-dd97-4291-a119-5f21033abc2d admin admin] The token cache is being invalidate because project a9fcf2dfafab4113881d38748b0824cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:50:19.845184 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c5ca4a75-dd97-4291-a119-5f21033abc2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:19.846068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c5ca4a75-dd97-4291-a119-5f21033abc2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:50:19.847848 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1354/2677] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:50:19 2022] DELETE /identity/v3/projects/a9fcf2dfafab4113881d38748b0824cb => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:50:19.854010 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8b929-2655-46de-ae17-9cb8bfe8fb92 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:19.854636 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8b929-2655-46de-ae17-9cb8bfe8fb92 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:19.854636 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8b929-2655-46de-ae17-9cb8bfe8fb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:19.874933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ccd8b929-2655-46de-ae17-9cb8bfe8fb92 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:19.878252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ccd8b929-2655-46de-ae17-9cb8bfe8fb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:19.960071 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1324/2678] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:19.969670 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0227be-6783-4726-b810-5b061b4ee4f3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:19.969964 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0227be-6783-4726-b810-5b061b4ee4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:19.970249 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0227be-6783-4726-b810-5b061b4ee4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:19.995484 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-bb0227be-6783-4726-b810-5b061b4ee4f3 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:19.996711 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bb0227be-6783-4726-b810-5b061b4ee4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:20.005047 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1355/2679] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:20.010634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8b929-2655-46de-ae17-9cb8bfe8fb92 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:20.021114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8b929-2655-46de-ae17-9cb8bfe8fb92 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:20.026127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8feb98-cd3f-4cf8-b3fd-233ffde26451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:20.026799 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8feb98-cd3f-4cf8-b3fd-233ffde26451 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:20.027090 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8feb98-cd3f-4cf8-b3fd-233ffde26451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:20.028295 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8feb98-cd3f-4cf8-b3fd-233ffde26451 admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:20.029052 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8feb98-cd3f-4cf8-b3fd-233ffde26451 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:50:20.029402 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8feb98-cd3f-4cf8-b3fd-233ffde26451 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:20.031040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8feb98-cd3f-4cf8-b3fd-233ffde26451 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:20.050998 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1325/2680] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:50:20.056565 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0227be-6783-4726-b810-5b061b4ee4f3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:20.071210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0227be-6783-4726-b810-5b061b4ee4f3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:20.077000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ec889-e8a2-4ce0-aa26-a8edfddd1822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:20.077710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ec889-e8a2-4ce0-aa26-a8edfddd1822 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:20.078002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ec889-e8a2-4ce0-aa26-a8edfddd1822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:20.078292 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ec889-e8a2-4ce0-aa26-a8edfddd1822 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:20.078992 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ec889-e8a2-4ce0-aa26-a8edfddd1822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:20.080909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ec889-e8a2-4ce0-aa26-a8edfddd1822 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:20.177010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1356/2681] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:50:20 2022] POST /identity/v3/projects => generated 369 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:50:20.182342 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8feb98-cd3f-4cf8-b3fd-233ffde26451 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:20.193920 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8feb98-cd3f-4cf8-b3fd-233ffde26451 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:20.261027 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837545b2-5759-4b9b-acde-a395b4b4a9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:20.261936 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837545b2-5759-4b9b-acde-a395b4b4a9ef admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:20.262235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837545b2-5759-4b9b-acde-a395b4b4a9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:20.262523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837545b2-5759-4b9b-acde-a395b4b4a9ef admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:20.263258 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837545b2-5759-4b9b-acde-a395b4b4a9ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:20.265220 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837545b2-5759-4b9b-acde-a395b4b4a9ef admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:20.335921 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1326/2682] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:50:20 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:50:20.344516 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ec889-e8a2-4ce0-aa26-a8edfddd1822 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:20.348360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:20.353190 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ec889-e8a2-4ce0-aa26-a8edfddd1822 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:20.357934 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c7201-bbc9-4eb5-8288-e8c0047aa5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:20.358664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c7201-bbc9-4eb5-8288-e8c0047aa5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:20.358976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c7201-bbc9-4eb5-8288-e8c0047aa5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:20.359234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c7201-bbc9-4eb5-8288-e8c0047aa5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:20.359959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c7201-bbc9-4eb5-8288-e8c0047aa5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:20.361656 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c7201-bbc9-4eb5-8288-e8c0047aa5c8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:20.373427 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1357/2683] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:50:20.377741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837545b2-5759-4b9b-acde-a395b4b4a9ef admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:20.396938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837545b2-5759-4b9b-acde-a395b4b4a9ef admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:20.404718 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f35fb3-e858-4513-9f74-26de2e15eac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:20.405439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f35fb3-e858-4513-9f74-26de2e15eac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:20.405725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f35fb3-e858-4513-9f74-26de2e15eac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:20.406006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f35fb3-e858-4513-9f74-26de2e15eac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:20.406750 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f35fb3-e858-4513-9f74-26de2e15eac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:20.408439 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f35fb3-e858-4513-9f74-26de2e15eac8 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:20.419244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1327/2684] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:50:20.423208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c7201-bbc9-4eb5-8288-e8c0047aa5c8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:20.437613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c7201-bbc9-4eb5-8288-e8c0047aa5c8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:20.439421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d88ba-e252-466f-afb8-73a332da9367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:20.440305 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d88ba-e252-466f-afb8-73a332da9367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:20.440591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d88ba-e252-466f-afb8-73a332da9367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:20.440869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d88ba-e252-466f-afb8-73a332da9367 admin admin] PATH_INFO: `/v3/projects/267667f0d657479bb538741450b97e3c/users/2c1d0c3e1e5048b9b7d826ac2d7d3b29/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:20.458183 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d88ba-e252-466f-afb8-73a332da9367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=267667f0d657479bb538741450b97e3c, user_id=2c1d0c3e1e5048b9b7d826ac2d7d3b29, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:20.460890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d88ba-e252-466f-afb8-73a332da9367 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:20.530187 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1358/2685] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:50:20 2022] PUT /identity/v3/projects/267667f0d657479bb538741450b97e3c/users/2c1d0c3e1e5048b9b7d826ac2d7d3b29/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:50:20.547429 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf81ae1-2473-45de-b993-46ad749be4bf None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:20.547726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf81ae1-2473-45de-b993-46ad749be4bf None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:20.548012 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf81ae1-2473-45de-b993-46ad749be4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:20.569237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:20.578088 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-4cf81ae1-2473-45de-b993-46ad749be4bf None None] MFA Rules not processed for user `2c1d0c3e1e5048b9b7d826ac2d7d3b29`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:20.579341 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4cf81ae1-2473-45de-b993-46ad749be4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:20.645649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1328/2686] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:50:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:20.658971 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d88ba-e252-466f-afb8-73a332da9367 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:20.660821 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-804d88ba-e252-466f-afb8-73a332da9367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:20.721443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d88ba-e252-466f-afb8-73a332da9367 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:20.725028 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30546af1-a493-4613-bc61-998ab1134768 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee7a6c342548f98e7dc73f4f71a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:20.725703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30546af1-a493-4613-bc61-998ab1134768 service glance] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:20.725991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30546af1-a493-4613-bc61-998ab1134768 service glance] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:20.726271 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30546af1-a493-4613-bc61-998ab1134768 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:20.734489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30546af1-a493-4613-bc61-998ab1134768 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:20.735910 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30546af1-a493-4613-bc61-998ab1134768 service glance] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:20.745911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1359/2687] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:50:20 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:50:26.368664 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2c334-3e92-4eb5-b5c8-d82302a4563f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:26.369253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2c334-3e92-4eb5-b5c8-d82302a4563f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:26.369365 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2c334-3e92-4eb5-b5c8-d82302a4563f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:26.388605 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-07b2c334-3e92-4eb5-b5c8-d82302a4563f None None] MFA Rules not processed for user `ba0229c6732541b4a6dda9040c038686`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:26.388753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-07b2c334-3e92-4eb5-b5c8-d82302a4563f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:26.447651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1329/2688] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:30.740720 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:36.440330 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=52 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:07.540750 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30546af1-a493-4613-bc61-998ab1134768 service glance] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:07.606047 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30546af1-a493-4613-bc61-998ab1134768 service glance] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:07.611262 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c8bb0-e114-488c-a031-55560c675816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:07.612157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c8bb0-e114-488c-a031-55560c675816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:07.612446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c8bb0-e114-488c-a031-55560c675816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:07.612734 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c8bb0-e114-488c-a031-55560c675816 admin admin] PATH_INFO: `/v3/users/2c1d0c3e1e5048b9b7d826ac2d7d3b29` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:07.613982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c8bb0-e114-488c-a031-55560c675816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c1d0c3e1e5048b9b7d826ac2d7d3b29)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:07.615903 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c8bb0-e114-488c-a031-55560c675816 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:07.821867 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-044c8bb0-e114-488c-a031-55560c675816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c1d0c3e1e5048b9b7d826ac2d7d3b29'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:07.834255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-044c8bb0-e114-488c-a031-55560c675816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c1d0c3e1e5048b9b7d826ac2d7d3b29'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:07.905493 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-044c8bb0-e114-488c-a031-55560c675816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c1d0c3e1e5048b9b7d826ac2d7d3b29'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:07.906055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-044c8bb0-e114-488c-a031-55560c675816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c1d0c3e1e5048b9b7d826ac2d7d3b29'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:07.914231 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-044c8bb0-e114-488c-a031-55560c675816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c1d0c3e1e5048b9b7d826ac2d7d3b29'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:07.920147 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1360/2689] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:51:07 2022] DELETE /identity/v3/users/2c1d0c3e1e5048b9b7d826ac2d7d3b29 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:51:07.925723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cd547-eea3-4fa5-8d71-9a138b8d5396 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:07.928165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cd547-eea3-4fa5-8d71-9a138b8d5396 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:07.928165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cd547-eea3-4fa5-8d71-9a138b8d5396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:07.943638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6b2cd547-eea3-4fa5-8d71-9a138b8d5396 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:51:07.944914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6b2cd547-eea3-4fa5-8d71-9a138b8d5396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:08.002660 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1330/2690] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:51:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:51:08.033380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c8bb0-e114-488c-a031-55560c675816 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:08.034843 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-044c8bb0-e114-488c-a031-55560c675816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:08.093137 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c8bb0-e114-488c-a031-55560c675816 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:08.097325 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed2707-bdf3-4c53-852a-627685c75607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:08.098000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2707-bdf3-4c53-852a-627685c75607 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:08.098278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2707-bdf3-4c53-852a-627685c75607 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:08.098559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2707-bdf3-4c53-852a-627685c75607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:08.108092 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed2707-bdf3-4c53-852a-627685c75607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:08.109614 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed2707-bdf3-4c53-852a-627685c75607 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:08.120056 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1361/2691] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:51:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:51:08.709543 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cd547-eea3-4fa5-8d71-9a138b8d5396 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:08.711282 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6b2cd547-eea3-4fa5-8d71-9a138b8d5396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:08.724643 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cd547-eea3-4fa5-8d71-9a138b8d5396 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:08.729952 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d25c5-5760-4144-9173-911e57c7942f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:08.730916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d25c5-5760-4144-9173-911e57c7942f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:08.731271 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d25c5-5760-4144-9173-911e57c7942f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:08.731650 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d25c5-5760-4144-9173-911e57c7942f admin admin] PATH_INFO: `/v3/projects/267667f0d657479bb538741450b97e3c` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:08.733021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d25c5-5760-4144-9173-911e57c7942f admin admin] RBAC: Authorizing `identity:delete_project(project_id=267667f0d657479bb538741450b97e3c)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:08.735250 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d25c5-5760-4144-9173-911e57c7942f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:08.855908 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-983d25c5-5760-4144-9173-911e57c7942f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '267667f0d657479bb538741450b97e3c', 'request_id': 'req-983d25c5-5760-4144-9173-911e57c7942f'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:08.861618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-983d25c5-5760-4144-9173-911e57c7942f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '267667f0d657479bb538741450b97e3c', 'request_id': 'req-983d25c5-5760-4144-9173-911e57c7942f'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:08.867662 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-983d25c5-5760-4144-9173-911e57c7942f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '267667f0d657479bb538741450b97e3c', 'request_id': 'req-983d25c5-5760-4144-9173-911e57c7942f'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:08.945710 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-983d25c5-5760-4144-9173-911e57c7942f admin admin] The token cache is being invalidate because project 267667f0d657479bb538741450b97e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:51:08.946422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-983d25c5-5760-4144-9173-911e57c7942f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:08.946627 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-983d25c5-5760-4144-9173-911e57c7942f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:08.948184 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1331/2692] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:51:08 2022] DELETE /identity/v3/projects/267667f0d657479bb538741450b97e3c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:51:08.953110 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3226162-c7c5-4822-b1c1-77e3444115ac None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:08.953563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3226162-c7c5-4822-b1c1-77e3444115ac None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:08.953735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3226162-c7c5-4822-b1c1-77e3444115ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:08.971134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d3226162-c7c5-4822-b1c1-77e3444115ac None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:51:08.972356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d3226162-c7c5-4822-b1c1-77e3444115ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:08.989633 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:09.031269 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1362/2693] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:51:09.039603 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d1e35-d2e7-4bb5-859e-f4c69b852ef2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:09.039917 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d1e35-d2e7-4bb5-859e-f4c69b852ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:09.040180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d1e35-d2e7-4bb5-859e-f4c69b852ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:09.059623 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-134d1e35-d2e7-4bb5-859e-f4c69b852ef2 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:51:09.060970 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-134d1e35-d2e7-4bb5-859e-f4c69b852ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:09.069826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1332/2694] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:51:09.074140 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3226162-c7c5-4822-b1c1-77e3444115ac None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:09.084748 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3226162-c7c5-4822-b1c1-77e3444115ac None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:09.089333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c352a9d-6d6b-4e3b-837c-e0ddbaac3941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:09.089993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c352a9d-6d6b-4e3b-837c-e0ddbaac3941 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:09.090281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c352a9d-6d6b-4e3b-837c-e0ddbaac3941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:09.090563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c352a9d-6d6b-4e3b-837c-e0ddbaac3941 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:09.091287 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c352a9d-6d6b-4e3b-837c-e0ddbaac3941 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:51:09.091596 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c352a9d-6d6b-4e3b-837c-e0ddbaac3941 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:09.093214 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c352a9d-6d6b-4e3b-837c-e0ddbaac3941 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:09.105053 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1363/2695] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:51:09.112849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d1e35-d2e7-4bb5-859e-f4c69b852ef2 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:09.119118 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d1e35-d2e7-4bb5-859e-f4c69b852ef2 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:09.124414 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3039ed-6dcf-49bc-9de1-80275f86b69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:09.125082 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3039ed-6dcf-49bc-9de1-80275f86b69f admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:09.125378 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3039ed-6dcf-49bc-9de1-80275f86b69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:09.126352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3039ed-6dcf-49bc-9de1-80275f86b69f admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:09.127143 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3039ed-6dcf-49bc-9de1-80275f86b69f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:09.129048 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3039ed-6dcf-49bc-9de1-80275f86b69f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:09.209329 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1333/2696] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:51:09 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:51:09.213230 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c352a9d-6d6b-4e3b-837c-e0ddbaac3941 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:09.223883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c352a9d-6d6b-4e3b-837c-e0ddbaac3941 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:09.280662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ea9f1-2d5a-45b4-b82f-3a758effb22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:09.281366 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ea9f1-2d5a-45b4-b82f-3a758effb22c admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:09.281661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ea9f1-2d5a-45b4-b82f-3a758effb22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:09.282460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ea9f1-2d5a-45b4-b82f-3a758effb22c admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:09.283160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ea9f1-2d5a-45b4-b82f-3a758effb22c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:09.285141 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ea9f1-2d5a-45b4-b82f-3a758effb22c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:09.390534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1364/2697] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:51:09 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:51:09.394549 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3039ed-6dcf-49bc-9de1-80275f86b69f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:09.405119 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3039ed-6dcf-49bc-9de1-80275f86b69f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:09.410695 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e75f6c-a282-4fed-a426-5cc063c83acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:09.413660 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e75f6c-a282-4fed-a426-5cc063c83acf admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:09.413660 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e75f6c-a282-4fed-a426-5cc063c83acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:09.413660 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e75f6c-a282-4fed-a426-5cc063c83acf admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:09.413660 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e75f6c-a282-4fed-a426-5cc063c83acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:09.414374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e75f6c-a282-4fed-a426-5cc063c83acf admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:09.425914 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1334/2698] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:51:09.429836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ea9f1-2d5a-45b4-b82f-3a758effb22c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:09.440271 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ea9f1-2d5a-45b4-b82f-3a758effb22c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:09.444883 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f29972-9e18-4e0d-8442-5325661ef629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:09.445574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f29972-9e18-4e0d-8442-5325661ef629 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:09.448089 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f29972-9e18-4e0d-8442-5325661ef629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:09.448089 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f29972-9e18-4e0d-8442-5325661ef629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:09.448089 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f29972-9e18-4e0d-8442-5325661ef629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:09.448400 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f29972-9e18-4e0d-8442-5325661ef629 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:09.458621 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1365/2699] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:51:09.462740 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e75f6c-a282-4fed-a426-5cc063c83acf admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:09.473263 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e75f6c-a282-4fed-a426-5cc063c83acf admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:09.477828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf4aec-afa2-439d-a51b-73f3950108f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:09.478697 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf4aec-afa2-439d-a51b-73f3950108f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:09.478977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf4aec-afa2-439d-a51b-73f3950108f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:09.479252 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf4aec-afa2-439d-a51b-73f3950108f2 admin admin] PATH_INFO: `/v3/projects/4c1b36ef4dfe4aab851fe6c0d2680fe0/users/3cb19028337347329f21f278df8e98a7/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:09.494348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf4aec-afa2-439d-a51b-73f3950108f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1b36ef4dfe4aab851fe6c0d2680fe0, user_id=3cb19028337347329f21f278df8e98a7, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:09.497037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf4aec-afa2-439d-a51b-73f3950108f2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:09.580805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1335/2700] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:51:09 2022] PUT /identity/v3/projects/4c1b36ef4dfe4aab851fe6c0d2680fe0/users/3cb19028337347329f21f278df8e98a7/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:51:09.594669 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8e65b-f52c-42d9-a31c-f65b1b1e267b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:09.595463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8e65b-f52c-42d9-a31c-f65b1b1e267b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:09.595768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8e65b-f52c-42d9-a31c-f65b1b1e267b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:09.627873 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-daa8e65b-f52c-42d9-a31c-f65b1b1e267b None None] MFA Rules not processed for user `3cb19028337347329f21f278df8e98a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:51:09.629085 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-daa8e65b-f52c-42d9-a31c-f65b1b1e267b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:09.702709 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1366/2701] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:51:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:51:09.717661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf4aec-afa2-439d-a51b-73f3950108f2 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:09.717661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7ebf4aec-afa2-439d-a51b-73f3950108f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:09.743996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:09.775947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf4aec-afa2-439d-a51b-73f3950108f2 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:09.779683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cbff6-2063-4ed3-b6c0-23aedf0ff753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:09.780370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cbff6-2063-4ed3-b6c0-23aedf0ff753 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:09.780663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cbff6-2063-4ed3-b6c0-23aedf0ff753 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:09.780949 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cbff6-2063-4ed3-b6c0-23aedf0ff753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:09.791026 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3cbff6-2063-4ed3-b6c0-23aedf0ff753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:09.795759 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3cbff6-2063-4ed3-b6c0-23aedf0ff753 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:09.802756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1336/2702] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:51:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:51:11.646864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e917d-2298-4966-a90e-0b7730780e86 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:11.647521 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e917d-2298-4966-a90e-0b7730780e86 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:11.647521 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e917d-2298-4966-a90e-0b7730780e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:11.664834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e81e917d-2298-4966-a90e-0b7730780e86 None None] MFA Rules not processed for user `3cb19028337347329f21f278df8e98a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:51:11.666080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e81e917d-2298-4966-a90e-0b7730780e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:11.672899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1367/2703] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:51:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:51:20.104403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=17 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:21.072392 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9651e6-ae6e-4083-a11f-292950bd1c8f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:21.072688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9651e6-ae6e-4083-a11f-292950bd1c8f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:21.073002 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9651e6-ae6e-4083-a11f-292950bd1c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:21.091683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ad9651e6-ae6e-4083-a11f-292950bd1c8f None None] MFA Rules not processed for user `6c14e93cced14b1b95e2de8fedce31e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:51:21.092889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ad9651e6-ae6e-4083-a11f-292950bd1c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:21.145842 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1337/2704] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:51:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:51:21.659688 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:31.139152 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=25 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:34.347575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e917d-2298-4966-a90e-0b7730780e86 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:34.350182 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e81e917d-2298-4966-a90e-0b7730780e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:34.437079 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e917d-2298-4966-a90e-0b7730780e86 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:34.441205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f0898-4932-4578-98da-487516219bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:34.441916 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f0898-4932-4578-98da-487516219bfb service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:34.442201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f0898-4932-4578-98da-487516219bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:34.442497 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f0898-4932-4578-98da-487516219bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:34.444465 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9f0f0898-4932-4578-98da-487516219bfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:34.517307 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f0898-4932-4578-98da-487516219bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:34.518773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f0898-4932-4578-98da-487516219bfb service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:34.529821 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1368/2705] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:51:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:51:35.182584 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48560314-93c8-4512-ad7e-2fd6578d987f None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:35.183114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48560314-93c8-4512-ad7e-2fd6578d987f None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:35.183114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48560314-93c8-4512-ad7e-2fd6578d987f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:35.203206 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-48560314-93c8-4512-ad7e-2fd6578d987f None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:51:35.205829 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-48560314-93c8-4512-ad7e-2fd6578d987f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:35.262182 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1338/2706] 10.222.0.64 () {58 vars in 1009 bytes} [Mon May 23 15:51:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:51:35.406156 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f0898-4932-4578-98da-487516219bfb service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:35.409083 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9f0f0898-4932-4578-98da-487516219bfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:35.482122 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f0898-4932-4578-98da-487516219bfb service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:35.487245 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec14297-4edf-4fea-9d86-bacf851f344e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:35.487892 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec14297-4edf-4fea-9d86-bacf851f344e service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:35.488177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec14297-4edf-4fea-9d86-bacf851f344e service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:35.488455 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec14297-4edf-4fea-9d86-bacf851f344e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:35.496835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec14297-4edf-4fea-9d86-bacf851f344e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:35.498550 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec14297-4edf-4fea-9d86-bacf851f344e service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:35.507539 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1369/2707] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:51:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:51:35.756076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48560314-93c8-4512-ad7e-2fd6578d987f None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:35.766671 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48560314-93c8-4512-ad7e-2fd6578d987f None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:35.770716 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86758a54-94a8-4efc-9bce-2b3c742fa2a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:35.771762 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86758a54-94a8-4efc-9bce-2b3c742fa2a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:35.772285 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86758a54-94a8-4efc-9bce-2b3c742fa2a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:35.772753 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86758a54-94a8-4efc-9bce-2b3c742fa2a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:35.774816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-86758a54-94a8-4efc-9bce-2b3c742fa2a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:35.793922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86758a54-94a8-4efc-9bce-2b3c742fa2a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:35.795577 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86758a54-94a8-4efc-9bce-2b3c742fa2a9 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:35.811180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1339/2708] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:51:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:51:36.275018 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c6b26-8c39-4209-ba9c-6c9525319659 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:36.276075 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c6b26-8c39-4209-ba9c-6c9525319659 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:36.276600 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c6b26-8c39-4209-ba9c-6c9525319659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:36.303692 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-895c6b26-8c39-4209-ba9c-6c9525319659 None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:51:36.305282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-895c6b26-8c39-4209-ba9c-6c9525319659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:36.316882 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1370/2709] 10.222.0.64 () {58 vars in 1009 bytes} [Mon May 23 15:51:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:51:36.333724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86758a54-94a8-4efc-9bce-2b3c742fa2a9 service neutron] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:36.351107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86758a54-94a8-4efc-9bce-2b3c742fa2a9 service neutron] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:36.363287 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e5230-f83a-4cb5-8fea-ba4ec8261bc5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:36.364013 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e5230-f83a-4cb5-8fea-ba4ec8261bc5 service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:36.364304 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e5230-f83a-4cb5-8fea-ba4ec8261bc5 service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:36.364586 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e5230-f83a-4cb5-8fea-ba4ec8261bc5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:36.384174 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e5230-f83a-4cb5-8fea-ba4ec8261bc5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:36.386060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e5230-f83a-4cb5-8fea-ba4ec8261bc5 service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:36.401947 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1340/2710] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:51:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:51:37.398095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6723d-21f6-4bda-b7a7-7c0da6511a1c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:37.401537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6723d-21f6-4bda-b7a7-7c0da6511a1c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:37.401537 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6723d-21f6-4bda-b7a7-7c0da6511a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:37.422286 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-84e6723d-21f6-4bda-b7a7-7c0da6511a1c None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:51:37.423920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-84e6723d-21f6-4bda-b7a7-7c0da6511a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:37.434951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1371/2711] 10.222.0.64 () {58 vars in 967 bytes} [Mon May 23 15:51:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:51:37.460015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e5230-f83a-4cb5-8fea-ba4ec8261bc5 service swift] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:37.465091 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e5230-f83a-4cb5-8fea-ba4ec8261bc5 service swift] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:37.471517 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35c8a2-4bef-463e-ae67-4d4ace464d87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:37.472421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35c8a2-4bef-463e-ae67-4d4ace464d87 service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:37.472899 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35c8a2-4bef-463e-ae67-4d4ace464d87 service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:37.473374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35c8a2-4bef-463e-ae67-4d4ace464d87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:37.483036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35c8a2-4bef-463e-ae67-4d4ace464d87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:37.488344 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35c8a2-4bef-463e-ae67-4d4ace464d87 service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:37.495877 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1341/2712] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:51:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:51:38.251570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9ba38-62cd-44fb-b662-02ee29519f25 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:38.251862 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9ba38-62cd-44fb-b662-02ee29519f25 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:38.252134 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9ba38-62cd-44fb-b662-02ee29519f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:38.283036 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-63c9ba38-62cd-44fb-b662-02ee29519f25 None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:51:38.284281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-63c9ba38-62cd-44fb-b662-02ee29519f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:38.292630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1372/2713] 10.222.0.64 () {58 vars in 1009 bytes} [Mon May 23 15:51:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:51:38.309516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35c8a2-4bef-463e-ae67-4d4ace464d87 service swift] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:38.327381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35c8a2-4bef-463e-ae67-4d4ace464d87 service swift] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:38.333977 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fed47-4727-45c8-b735-de88e7ef6922 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:38.334803 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fed47-4727-45c8-b735-de88e7ef6922 service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:38.335160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fed47-4727-45c8-b735-de88e7ef6922 service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:38.335643 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fed47-4727-45c8-b735-de88e7ef6922 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:38.352000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fed47-4727-45c8-b735-de88e7ef6922 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:38.354032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fed47-4727-45c8-b735-de88e7ef6922 service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:38.368440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1342/2714] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:51:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:51:39.503450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d417b5b-8fbd-4f68-9855-a82291cd3465 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:39.504157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d417b5b-8fbd-4f68-9855-a82291cd3465 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:39.504683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d417b5b-8fbd-4f68-9855-a82291cd3465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:39.527019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8d417b5b-8fbd-4f68-9855-a82291cd3465 None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:51:39.528775 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8d417b5b-8fbd-4f68-9855-a82291cd3465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:39.541094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1373/2715] 10.222.0.64 () {58 vars in 967 bytes} [Mon May 23 15:51:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:51:39.560398 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fed47-4727-45c8-b735-de88e7ef6922 service swift] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:39.572370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fed47-4727-45c8-b735-de88e7ef6922 service swift] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:39.578896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495cabc-c6df-433a-bbf1-98fa9c0f8309 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:39.580008 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495cabc-c6df-433a-bbf1-98fa9c0f8309 service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:39.580501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495cabc-c6df-433a-bbf1-98fa9c0f8309 service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:39.581006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495cabc-c6df-433a-bbf1-98fa9c0f8309 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:39.590852 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495cabc-c6df-433a-bbf1-98fa9c0f8309 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:39.592798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495cabc-c6df-433a-bbf1-98fa9c0f8309 service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:39.604022 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1343/2716] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:51:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:51:42.568557 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d417b5b-8fbd-4f68-9855-a82291cd3465 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:42.574917 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:42.643505 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d417b5b-8fbd-4f68-9855-a82291cd3465 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:42.648264 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eac1d5-2f77-4b6e-afeb-0f955ac623a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:42.649435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eac1d5-2f77-4b6e-afeb-0f955ac623a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:42.650381 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eac1d5-2f77-4b6e-afeb-0f955ac623a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:42.650820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eac1d5-2f77-4b6e-afeb-0f955ac623a5 admin admin] PATH_INFO: `/v3/users/3cb19028337347329f21f278df8e98a7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:42.653662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eac1d5-2f77-4b6e-afeb-0f955ac623a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cb19028337347329f21f278df8e98a7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:42.655612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eac1d5-2f77-4b6e-afeb-0f955ac623a5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:42.863076 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-37eac1d5-2f77-4b6e-afeb-0f955ac623a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cb19028337347329f21f278df8e98a7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:42.885701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-37eac1d5-2f77-4b6e-afeb-0f955ac623a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cb19028337347329f21f278df8e98a7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:43.024779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-37eac1d5-2f77-4b6e-afeb-0f955ac623a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cb19028337347329f21f278df8e98a7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:43.026768 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-37eac1d5-2f77-4b6e-afeb-0f955ac623a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cb19028337347329f21f278df8e98a7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:43.037687 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-37eac1d5-2f77-4b6e-afeb-0f955ac623a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cb19028337347329f21f278df8e98a7'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:43.049152 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1374/2717] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:51:42 2022] DELETE /identity/v3/users/3cb19028337347329f21f278df8e98a7 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:51:43.050610 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c56c7-e0b1-40ca-a53f-166c7611da64 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:43.051102 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c56c7-e0b1-40ca-a53f-166c7611da64 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:43.051221 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c56c7-e0b1-40ca-a53f-166c7611da64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:43.075122 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ea5c56c7-e0b1-40ca-a53f-166c7611da64 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:51:43.083879 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ea5c56c7-e0b1-40ca-a53f-166c7611da64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:43.155214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1344/2718] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:51:43.167240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eac1d5-2f77-4b6e-afeb-0f955ac623a5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:43.168947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-37eac1d5-2f77-4b6e-afeb-0f955ac623a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:43.264374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eac1d5-2f77-4b6e-afeb-0f955ac623a5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:43.268968 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618a74f7-ac58-4439-b94a-546ad3a93d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:43.269920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a74f7-ac58-4439-b94a-546ad3a93d98 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:43.270375 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a74f7-ac58-4439-b94a-546ad3a93d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:43.270820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a74f7-ac58-4439-b94a-546ad3a93d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:43.287657 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618a74f7-ac58-4439-b94a-546ad3a93d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:43.287657 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618a74f7-ac58-4439-b94a-546ad3a93d98 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:43.301413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1375/2719] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:51:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:51:43.638758 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c56c7-e0b1-40ca-a53f-166c7611da64 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:43.646759 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ea5c56c7-e0b1-40ca-a53f-166c7611da64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:43.682768 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c56c7-e0b1-40ca-a53f-166c7611da64 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:43.688956 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7b0a2-52ef-47a5-9014-e898aa20b056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:43.693479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7b0a2-52ef-47a5-9014-e898aa20b056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:43.693479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7b0a2-52ef-47a5-9014-e898aa20b056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:43.693479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7b0a2-52ef-47a5-9014-e898aa20b056 admin admin] PATH_INFO: `/v3/projects/4c1b36ef4dfe4aab851fe6c0d2680fe0` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:43.693479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7b0a2-52ef-47a5-9014-e898aa20b056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c1b36ef4dfe4aab851fe6c0d2680fe0)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:43.694120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7b0a2-52ef-47a5-9014-e898aa20b056 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:43.806562 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:43.820803 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52e7b0a2-52ef-47a5-9014-e898aa20b056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c1b36ef4dfe4aab851fe6c0d2680fe0', 'request_id': 'req-52e7b0a2-52ef-47a5-9014-e898aa20b056'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:43.827028 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52e7b0a2-52ef-47a5-9014-e898aa20b056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c1b36ef4dfe4aab851fe6c0d2680fe0', 'request_id': 'req-52e7b0a2-52ef-47a5-9014-e898aa20b056'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:43.841545 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52e7b0a2-52ef-47a5-9014-e898aa20b056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c1b36ef4dfe4aab851fe6c0d2680fe0', 'request_id': 'req-52e7b0a2-52ef-47a5-9014-e898aa20b056'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:43.916615 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52e7b0a2-52ef-47a5-9014-e898aa20b056 admin admin] The token cache is being invalidate because project 4c1b36ef4dfe4aab851fe6c0d2680fe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:51:43.916820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52e7b0a2-52ef-47a5-9014-e898aa20b056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:43.920234 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-52e7b0a2-52ef-47a5-9014-e898aa20b056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:51:43.938501 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1345/2720] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:51:43 2022] DELETE /identity/v3/projects/4c1b36ef4dfe4aab851fe6c0d2680fe0 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:51:53.291704 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=77,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:53.948995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:59.252945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3157a9-f6f5-4fc2-ba4c-3ba8d7c5f1f1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:59.256218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3157a9-f6f5-4fc2-ba4c-3ba8d7c5f1f1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:59.256218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3157a9-f6f5-4fc2-ba4c-3ba8d7c5f1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:59.270853 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-dd3157a9-f6f5-4fc2-ba4c-3ba8d7c5f1f1 None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:51:59.272034 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dd3157a9-f6f5-4fc2-ba4c-3ba8d7c5f1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:59.326413 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1376/2721] 10.222.0.64 () {58 vars in 1009 bytes} [Mon May 23 15:51:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:51:59.339893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7b0a2-52ef-47a5-9014-e898aa20b056 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:51:59.341330 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-52e7b0a2-52ef-47a5-9014-e898aa20b056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:51:59.409803 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7b0a2-52ef-47a5-9014-e898aa20b056 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:51:59.415647 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c56e0a-f185-47ef-ba65-621425090e00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:51:59.416329 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c56e0a-f185-47ef-ba65-621425090e00 service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:51:59.416614 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c56e0a-f185-47ef-ba65-621425090e00 service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:51:59.416890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c56e0a-f185-47ef-ba65-621425090e00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:51:59.425376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c56e0a-f185-47ef-ba65-621425090e00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:51:59.433375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c56e0a-f185-47ef-ba65-621425090e00 service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:51:59.435757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1346/2722] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:51:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:52:09.319463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=25 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:09.431772 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=32 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:14.908968 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3157a9-f6f5-4fc2-ba4c-3ba8d7c5f1f1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:14.910574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dd3157a9-f6f5-4fc2-ba4c-3ba8d7c5f1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:14.966722 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3157a9-f6f5-4fc2-ba4c-3ba8d7c5f1f1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:14.970265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb6b82-a3e8-43bc-ac7d-4750e171c944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:14.970909 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb6b82-a3e8-43bc-ac7d-4750e171c944 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:14.971191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb6b82-a3e8-43bc-ac7d-4750e171c944 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:14.971469 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb6b82-a3e8-43bc-ac7d-4750e171c944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:14.973265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-43cb6b82-a3e8-43bc-ac7d-4750e171c944 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:14.982744 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb6b82-a3e8-43bc-ac7d-4750e171c944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:14.984157 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb6b82-a3e8-43bc-ac7d-4750e171c944 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:14.991981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1377/2723] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:52:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:52:16.170440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c56e0a-f185-47ef-ba65-621425090e00 service swift] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:16.172127 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f0c56e0a-f185-47ef-ba65-621425090e00 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:16.215526 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb6b82-a3e8-43bc-ac7d-4750e171c944 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:16.217296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-43cb6b82-a3e8-43bc-ac7d-4750e171c944 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:16.229194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c56e0a-f185-47ef-ba65-621425090e00 service swift] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:16.232718 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9d2dc-dbf7-4c34-b911-5be536859563 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee7a6c342548f98e7dc73f4f71a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:16.233473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9d2dc-dbf7-4c34-b911-5be536859563 service glance] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:16.233756 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9d2dc-dbf7-4c34-b911-5be536859563 service glance] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:16.234035 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9d2dc-dbf7-4c34-b911-5be536859563 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:16.235817 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a4a9d2dc-dbf7-4c34-b911-5be536859563 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:16.294881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9d2dc-dbf7-4c34-b911-5be536859563 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:16.296445 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9d2dc-dbf7-4c34-b911-5be536859563 service glance] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:16.298225 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb6b82-a3e8-43bc-ac7d-4750e171c944 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:16.306326 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1347/2724] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:52:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:52:16.309765 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ff354-3c2f-4301-9502-2bbd543e07d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:16.310660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ff354-3c2f-4301-9502-2bbd543e07d1 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:16.311342 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ff354-3c2f-4301-9502-2bbd543e07d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:16.311928 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ff354-3c2f-4301-9502-2bbd543e07d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:16.322733 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ff354-3c2f-4301-9502-2bbd543e07d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:16.324584 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ff354-3c2f-4301-9502-2bbd543e07d1 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:16.334888 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1378/2725] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:52:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:52:16.337791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f25f1-9bf6-470b-bd14-05904c610fed None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:16.338093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f25f1-9bf6-470b-bd14-05904c610fed None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:16.338368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f25f1-9bf6-470b-bd14-05904c610fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:16.356269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-526f25f1-9bf6-470b-bd14-05904c610fed None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:52:16.357494 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-526f25f1-9bf6-470b-bd14-05904c610fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:16.364568 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1348/2726] 10.222.0.64 () {58 vars in 967 bytes} [Mon May 23 15:52:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:52:16.376799 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ff354-3c2f-4301-9502-2bbd543e07d1 service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:16.389158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ff354-3c2f-4301-9502-2bbd543e07d1 service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:16.394727 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019218f4-8861-4215-9cfe-bed35318a3a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:16.395387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019218f4-8861-4215-9cfe-bed35318a3a4 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:16.395679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019218f4-8861-4215-9cfe-bed35318a3a4 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:16.395962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019218f4-8861-4215-9cfe-bed35318a3a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:16.405793 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019218f4-8861-4215-9cfe-bed35318a3a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:16.407383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019218f4-8861-4215-9cfe-bed35318a3a4 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:16.416261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1379/2727] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:52:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 15:52:26.351200 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=51 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:26.412990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=66 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:36.482658 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f25f1-9bf6-470b-bd14-05904c610fed None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:36.482658 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-526f25f1-9bf6-470b-bd14-05904c610fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:36.498319 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f25f1-9bf6-470b-bd14-05904c610fed None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:36.504100 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde66ae3-b9a6-47ed-b492-39c73794765b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:36.505254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde66ae3-b9a6-47ed-b492-39c73794765b service nova] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:36.505757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde66ae3-b9a6-47ed-b492-39c73794765b service nova] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:36.506108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde66ae3-b9a6-47ed-b492-39c73794765b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:36.508267 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fde66ae3-b9a6-47ed-b492-39c73794765b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:36.525896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde66ae3-b9a6-47ed-b492-39c73794765b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:36.527519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde66ae3-b9a6-47ed-b492-39c73794765b service nova] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:36.539658 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1349/2728] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:52:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:52:39.197108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019218f4-8861-4215-9cfe-bed35318a3a4 service swift] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:39.208894 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019218f4-8861-4215-9cfe-bed35318a3a4 service swift] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:39.215076 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6a44a-31ed-4775-afdf-683b974a1a14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:39.215717 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6a44a-31ed-4775-afdf-683b974a1a14 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:39.216000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6a44a-31ed-4775-afdf-683b974a1a14 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:39.216280 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6a44a-31ed-4775-afdf-683b974a1a14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:39.218064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6ef6a44a-31ed-4775-afdf-683b974a1a14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:39.230965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6a44a-31ed-4775-afdf-683b974a1a14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:39.232516 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6a44a-31ed-4775-afdf-683b974a1a14 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:39.247926 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1380/2729] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:52:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:52:40.723617 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde66ae3-b9a6-47ed-b492-39c73794765b service nova] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:40.725165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fde66ae3-b9a6-47ed-b492-39c73794765b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:40.787237 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde66ae3-b9a6-47ed-b492-39c73794765b service nova] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:40.792370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1a47c-ed39-4d38-931b-cb751ac4f993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:40.793261 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1a47c-ed39-4d38-931b-cb751ac4f993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:40.793556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1a47c-ed39-4d38-931b-cb751ac4f993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:40.793836 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1a47c-ed39-4d38-931b-cb751ac4f993 admin admin] PATH_INFO: `/v3/users/6c14e93cced14b1b95e2de8fedce31e8` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:40.795008 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1a47c-ed39-4d38-931b-cb751ac4f993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c14e93cced14b1b95e2de8fedce31e8)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:40.796932 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1a47c-ed39-4d38-931b-cb751ac4f993 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:40.934154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d1d1a47c-ed39-4d38-931b-cb751ac4f993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c14e93cced14b1b95e2de8fedce31e8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:40.934816 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d1d1a47c-ed39-4d38-931b-cb751ac4f993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c14e93cced14b1b95e2de8fedce31e8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:40.935474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d1d1a47c-ed39-4d38-931b-cb751ac4f993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c14e93cced14b1b95e2de8fedce31e8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:40.942651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d1d1a47c-ed39-4d38-931b-cb751ac4f993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c14e93cced14b1b95e2de8fedce31e8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:40.955378 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d1d1a47c-ed39-4d38-931b-cb751ac4f993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c14e93cced14b1b95e2de8fedce31e8'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:41.005822 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1350/2730] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:52:40 2022] DELETE /identity/v3/users/6c14e93cced14b1b95e2de8fedce31e8 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:52:41.236777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6a44a-31ed-4775-afdf-683b974a1a14 service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:41.248438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6a44a-31ed-4775-afdf-683b974a1a14 service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:41.309483 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8001e8-cc5d-42a2-abf7-d965dc7ab3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:41.310396 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8001e8-cc5d-42a2-abf7-d965dc7ab3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:41.310663 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8001e8-cc5d-42a2-abf7-d965dc7ab3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:41.310947 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8001e8-cc5d-42a2-abf7-d965dc7ab3cd admin admin] PATH_INFO: `/v3/projects/58c0e4fa5fa246f08bb61703a42e2c59` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:41.312145 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8001e8-cc5d-42a2-abf7-d965dc7ab3cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c0e4fa5fa246f08bb61703a42e2c59)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:41.314093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8001e8-cc5d-42a2-abf7-d965dc7ab3cd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:41.429620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3d8001e8-cc5d-42a2-abf7-d965dc7ab3cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c0e4fa5fa246f08bb61703a42e2c59', 'request_id': 'req-3d8001e8-cc5d-42a2-abf7-d965dc7ab3cd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:41.435949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3d8001e8-cc5d-42a2-abf7-d965dc7ab3cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c0e4fa5fa246f08bb61703a42e2c59', 'request_id': 'req-3d8001e8-cc5d-42a2-abf7-d965dc7ab3cd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:41.526525 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3d8001e8-cc5d-42a2-abf7-d965dc7ab3cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c0e4fa5fa246f08bb61703a42e2c59', 'request_id': 'req-3d8001e8-cc5d-42a2-abf7-d965dc7ab3cd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:41.536936 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3d8001e8-cc5d-42a2-abf7-d965dc7ab3cd admin admin] The token cache is being invalidate because project 58c0e4fa5fa246f08bb61703a42e2c59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:52:41.537254 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3d8001e8-cc5d-42a2-abf7-d965dc7ab3cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:41.537872 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3d8001e8-cc5d-42a2-abf7-d965dc7ab3cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:41.539414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1381/2731] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:52:41 2022] DELETE /identity/v3/projects/58c0e4fa5fa246f08bb61703a42e2c59 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:52:41.545096 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef782f9-fe9b-4962-a062-1cb54e3b4f48 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:41.545407 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef782f9-fe9b-4962-a062-1cb54e3b4f48 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:41.545696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef782f9-fe9b-4962-a062-1cb54e3b4f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:41.565199 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-cef782f9-fe9b-4962-a062-1cb54e3b4f48 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:52:41.566409 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cef782f9-fe9b-4962-a062-1cb54e3b4f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:41.637097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1351/2732] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:52:41.644971 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33497d02-45d8-4e7a-81ad-2bc8f5ab5a71 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:41.645312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33497d02-45d8-4e7a-81ad-2bc8f5ab5a71 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:41.645610 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33497d02-45d8-4e7a-81ad-2bc8f5ab5a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:41.664569 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-33497d02-45d8-4e7a-81ad-2bc8f5ab5a71 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:52:41.665820 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-33497d02-45d8-4e7a-81ad-2bc8f5ab5a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:41.673667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1382/2733] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:52:41.678491 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef782f9-fe9b-4962-a062-1cb54e3b4f48 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:41.687972 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef782f9-fe9b-4962-a062-1cb54e3b4f48 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:41.692519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbdb74-5c0c-4cfd-b584-9bd1e86899fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:41.693217 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbdb74-5c0c-4cfd-b584-9bd1e86899fd admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:41.693502 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbdb74-5c0c-4cfd-b584-9bd1e86899fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:41.693779 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbdb74-5c0c-4cfd-b584-9bd1e86899fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:41.694499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adbdb74-5c0c-4cfd-b584-9bd1e86899fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:52:41.694807 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adbdb74-5c0c-4cfd-b584-9bd1e86899fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:41.697474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adbdb74-5c0c-4cfd-b584-9bd1e86899fd admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:41.699262 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:41.710071 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1352/2734] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:52:41.714481 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33497d02-45d8-4e7a-81ad-2bc8f5ab5a71 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:41.723692 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33497d02-45d8-4e7a-81ad-2bc8f5ab5a71 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:41.728911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e50512-6aeb-4570-9382-080b189f1ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:41.729592 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e50512-6aeb-4570-9382-080b189f1ade admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:41.729880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e50512-6aeb-4570-9382-080b189f1ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:41.730161 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e50512-6aeb-4570-9382-080b189f1ade admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:41.730849 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e50512-6aeb-4570-9382-080b189f1ade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:41.732734 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e50512-6aeb-4570-9382-080b189f1ade admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:41.819678 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1383/2735] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:52:41 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:52:41.825976 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbdb74-5c0c-4cfd-b584-9bd1e86899fd admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:41.836042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbdb74-5c0c-4cfd-b584-9bd1e86899fd admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:41.893704 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac2d5a-3ac2-4bd4-9b39-7320a828f0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:41.894381 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac2d5a-3ac2-4bd4-9b39-7320a828f0a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:41.894672 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac2d5a-3ac2-4bd4-9b39-7320a828f0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:41.894958 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac2d5a-3ac2-4bd4-9b39-7320a828f0a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:41.895653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac2d5a-3ac2-4bd4-9b39-7320a828f0a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:41.897679 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac2d5a-3ac2-4bd4-9b39-7320a828f0a6 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:42.029922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1353/2736] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:52:41 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:52:42.033899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e50512-6aeb-4570-9382-080b189f1ade admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:42.044605 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e50512-6aeb-4570-9382-080b189f1ade admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:42.049073 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba58a92-4629-4109-9890-61ca45d453d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:42.049990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba58a92-4629-4109-9890-61ca45d453d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:42.050279 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba58a92-4629-4109-9890-61ca45d453d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:42.050551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba58a92-4629-4109-9890-61ca45d453d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:42.051266 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba58a92-4629-4109-9890-61ca45d453d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:42.052868 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba58a92-4629-4109-9890-61ca45d453d1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:42.063240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1384/2737] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:52:42.066974 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac2d5a-3ac2-4bd4-9b39-7320a828f0a6 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:42.077416 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac2d5a-3ac2-4bd4-9b39-7320a828f0a6 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:42.082049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa5359-073c-4d5a-b797-6a8941fa4781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:42.082729 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa5359-073c-4d5a-b797-6a8941fa4781 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:42.083040 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa5359-073c-4d5a-b797-6a8941fa4781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:42.083325 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa5359-073c-4d5a-b797-6a8941fa4781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:42.084051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa5359-073c-4d5a-b797-6a8941fa4781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:42.085765 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa5359-073c-4d5a-b797-6a8941fa4781 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:42.096487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1354/2738] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:52:42.101116 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba58a92-4629-4109-9890-61ca45d453d1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:42.111593 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba58a92-4629-4109-9890-61ca45d453d1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:42.116073 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612a63c-19b6-4dea-a107-2c4e283023e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:42.116938 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612a63c-19b6-4dea-a107-2c4e283023e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:42.117253 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612a63c-19b6-4dea-a107-2c4e283023e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:42.117532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612a63c-19b6-4dea-a107-2c4e283023e5 admin admin] PATH_INFO: `/v3/projects/3daaf17c3dac442f82831c7df811f511/users/172c28d985cb40409d563fad4fa2ad5c/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:42.132993 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612a63c-19b6-4dea-a107-2c4e283023e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3daaf17c3dac442f82831c7df811f511, user_id=172c28d985cb40409d563fad4fa2ad5c, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:42.135781 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612a63c-19b6-4dea-a107-2c4e283023e5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:42.212890 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1385/2739] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:52:42 2022] PUT /identity/v3/projects/3daaf17c3dac442f82831c7df811f511/users/172c28d985cb40409d563fad4fa2ad5c/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:52:42.217938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a895c-f7ea-4a65-8959-23af42ad4da0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:42.218238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a895c-f7ea-4a65-8959-23af42ad4da0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:42.218521 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a895c-f7ea-4a65-8959-23af42ad4da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:42.236993 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-cb7a895c-f7ea-4a65-8959-23af42ad4da0 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:52:42.241673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cb7a895c-f7ea-4a65-8959-23af42ad4da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:42.297467 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1355/2740] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:52:42.307614 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612a63c-19b6-4dea-a107-2c4e283023e5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:42.309356 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c612a63c-19b6-4dea-a107-2c4e283023e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:42.323675 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:42.368779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612a63c-19b6-4dea-a107-2c4e283023e5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:42.372542 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952a201-b61b-4794-9751-a5971b1b832a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:42.373231 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952a201-b61b-4794-9751-a5971b1b832a service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:42.373523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952a201-b61b-4794-9751-a5971b1b832a service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:42.373804 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952a201-b61b-4794-9751-a5971b1b832a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:42.382954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952a201-b61b-4794-9751-a5971b1b832a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:42.385533 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952a201-b61b-4794-9751-a5971b1b832a service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:42.394861 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1386/2741] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:52:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:52:48.399452 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60d491-d0cd-4ac8-8fa5-26954f0207d1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:48.399950 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60d491-d0cd-4ac8-8fa5-26954f0207d1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:48.400036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60d491-d0cd-4ac8-8fa5-26954f0207d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:48.443896 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-2c60d491-d0cd-4ac8-8fa5-26954f0207d1 None None] MFA Rules not processed for user `172c28d985cb40409d563fad4fa2ad5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:52:48.445298 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-2c60d491-d0cd-4ac8-8fa5-26954f0207d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:48.522538 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1356/2742] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:52:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:52:48.527865 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952a201-b61b-4794-9751-a5971b1b832a service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:48.538083 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952a201-b61b-4794-9751-a5971b1b832a service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:48.543023 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c35a5-0fab-4f21-8ee3-1468f1b386da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:48.543702 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c35a5-0fab-4f21-8ee3-1468f1b386da admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:48.543992 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c35a5-0fab-4f21-8ee3-1468f1b386da admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:48.544276 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c35a5-0fab-4f21-8ee3-1468f1b386da admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:48.544972 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c35a5-0fab-4f21-8ee3-1468f1b386da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:48.546924 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c35a5-0fab-4f21-8ee3-1468f1b386da admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:48.659745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1387/2743] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:52:48 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:52:48.665241 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60d491-d0cd-4ac8-8fa5-26954f0207d1 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:48.669826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:48.683567 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60d491-d0cd-4ac8-8fa5-26954f0207d1 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:48.751032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c0e6b-c890-43ff-bc94-af110a0b72e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:48.751709 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c0e6b-c890-43ff-bc94-af110a0b72e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:48.751994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c0e6b-c890-43ff-bc94-af110a0b72e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:48.752285 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c0e6b-c890-43ff-bc94-af110a0b72e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:48.752960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c0e6b-c890-43ff-bc94-af110a0b72e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:48.754944 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c0e6b-c890-43ff-bc94-af110a0b72e6 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:48.846645 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1357/2744] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:52:48 2022] POST /identity/v3/users => generated 299 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:52:48.850645 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c35a5-0fab-4f21-8ee3-1468f1b386da admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:48.861738 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c35a5-0fab-4f21-8ee3-1468f1b386da admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:48.866718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d84af5-3410-4c9b-ae32-ddb4f0852330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:48.867383 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d84af5-3410-4c9b-ae32-ddb4f0852330 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:48.867779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d84af5-3410-4c9b-ae32-ddb4f0852330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:48.868007 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d84af5-3410-4c9b-ae32-ddb4f0852330 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:48.868764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d84af5-3410-4c9b-ae32-ddb4f0852330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:48.871004 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d84af5-3410-4c9b-ae32-ddb4f0852330 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:48.881719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1388/2745] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:52:48.886059 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c0e6b-c890-43ff-bc94-af110a0b72e6 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:48.896094 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c0e6b-c890-43ff-bc94-af110a0b72e6 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:48.902038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee23b69-064b-4b05-a559-1902a1866a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:48.902906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee23b69-064b-4b05-a559-1902a1866a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:48.903203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee23b69-064b-4b05-a559-1902a1866a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:48.903461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee23b69-064b-4b05-a559-1902a1866a71 admin admin] PATH_INFO: `/v3/projects/8c7e2b79f5064252a47372718fe656f4/users/7080c136d70e40a7850e54fc8dabde05/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:48.920310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee23b69-064b-4b05-a559-1902a1866a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7e2b79f5064252a47372718fe656f4, user_id=7080c136d70e40a7850e54fc8dabde05, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:48.923367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee23b69-064b-4b05-a559-1902a1866a71 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:49.060862 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1358/2746] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:52:48 2022] PUT /identity/v3/projects/8c7e2b79f5064252a47372718fe656f4/users/7080c136d70e40a7850e54fc8dabde05/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:52:49.064759 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d84af5-3410-4c9b-ae32-ddb4f0852330 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:49.075844 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d84af5-3410-4c9b-ae32-ddb4f0852330 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:49.142369 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f9c80-0460-49ef-a76d-cf51d718017b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:49.143143 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f9c80-0460-49ef-a76d-cf51d718017b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:49.143480 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f9c80-0460-49ef-a76d-cf51d718017b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:49.143767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f9c80-0460-49ef-a76d-cf51d718017b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:49.144611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f9c80-0460-49ef-a76d-cf51d718017b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:49.146316 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f9c80-0460-49ef-a76d-cf51d718017b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:49.157762 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1389/2747] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:52:49.162008 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee23b69-064b-4b05-a559-1902a1866a71 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:49.174073 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee23b69-064b-4b05-a559-1902a1866a71 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:49.182154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3f878-ff64-41a8-9287-0c7ef80c8345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:49.182154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3f878-ff64-41a8-9287-0c7ef80c8345 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:49.182154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3f878-ff64-41a8-9287-0c7ef80c8345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:49.182154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3f878-ff64-41a8-9287-0c7ef80c8345 admin admin] PATH_INFO: `/v3/projects/8c7e2b79f5064252a47372718fe656f4/users/7080c136d70e40a7850e54fc8dabde05/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:49.184481 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d3f878-ff64-41a8-9287-0c7ef80c8345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7e2b79f5064252a47372718fe656f4, user_id=7080c136d70e40a7850e54fc8dabde05, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:49.187185 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d3f878-ff64-41a8-9287-0c7ef80c8345 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:49.198041 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1359/2748] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:52:49 2022] PUT /identity/v3/projects/8c7e2b79f5064252a47372718fe656f4/users/7080c136d70e40a7850e54fc8dabde05/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:52:53.142868 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f9c80-0460-49ef-a76d-cf51d718017b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:53.211836 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f9c80-0460-49ef-a76d-cf51d718017b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:53.264623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:53.303104 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d7cc3-44d3-4161-9171-9306f4df8e9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:53.303815 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d7cc3-44d3-4161-9171-9306f4df8e9f service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:53.304105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d7cc3-44d3-4161-9171-9306f4df8e9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:53.304389 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d7cc3-44d3-4161-9171-9306f4df8e9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:53.306460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a02d7cc3-44d3-4161-9171-9306f4df8e9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:53.393221 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d7cc3-44d3-4161-9171-9306f4df8e9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:53.395160 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d7cc3-44d3-4161-9171-9306f4df8e9f service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:53.409090 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1390/2749] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:52:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:52:56.098871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a24d17-aa9b-4280-9588-69318051d40a None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:56.099387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a24d17-aa9b-4280-9588-69318051d40a None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:56.103716 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a24d17-aa9b-4280-9588-69318051d40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:56.166689 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-94a24d17-aa9b-4280-9588-69318051d40a None None] MFA Rules not processed for user `7080c136d70e40a7850e54fc8dabde05`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:52:56.168328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-94a24d17-aa9b-4280-9588-69318051d40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:56.303308 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1360/2750] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:52:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:52:56.317671 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d7cc3-44d3-4161-9171-9306f4df8e9f service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:56.319970 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a02d7cc3-44d3-4161-9171-9306f4df8e9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:56.410327 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d7cc3-44d3-4161-9171-9306f4df8e9f service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:56.434579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97cdfb-708b-4bd9-adb0-57c3852f054c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee7a6c342548f98e7dc73f4f71a66d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:56.435611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97cdfb-708b-4bd9-adb0-57c3852f054c service glance] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:56.436136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97cdfb-708b-4bd9-adb0-57c3852f054c service glance] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:56.436621 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97cdfb-708b-4bd9-adb0-57c3852f054c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:56.555209 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97cdfb-708b-4bd9-adb0-57c3852f054c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:56.556713 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97cdfb-708b-4bd9-adb0-57c3852f054c service glance] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:56.592415 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1391/2751] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:52:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:52:56.971726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8d522-0d16-42d8-b8f7-bb946c720d89 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:56.971726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8d522-0d16-42d8-b8f7-bb946c720d89 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:56.971726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8d522-0d16-42d8-b8f7-bb946c720d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:57.015962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-5fe8d522-0d16-42d8-b8f7-bb946c720d89 None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:52:57.020051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5fe8d522-0d16-42d8-b8f7-bb946c720d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:57.061802 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:57.089536 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1361/2752] 10.222.0.64 () {58 vars in 1009 bytes} [Mon May 23 15:52:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:52:57.128419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97cdfb-708b-4bd9-adb0-57c3852f054c service glance] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:57.131861 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bc97cdfb-708b-4bd9-adb0-57c3852f054c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:57.248829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:57.277517 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97cdfb-708b-4bd9-adb0-57c3852f054c service glance] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:57.283653 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802d925-0d05-4e35-92b2-6b00742f2219 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:57.284614 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802d925-0d05-4e35-92b2-6b00742f2219 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:57.284785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802d925-0d05-4e35-92b2-6b00742f2219 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:57.285197 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802d925-0d05-4e35-92b2-6b00742f2219 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:57.295705 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802d925-0d05-4e35-92b2-6b00742f2219 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:57.297566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802d925-0d05-4e35-92b2-6b00742f2219 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:57.307805 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1392/2753] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:52:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:52:58.793909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8d522-0d16-42d8-b8f7-bb946c720d89 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:58.796619 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-5fe8d522-0d16-42d8-b8f7-bb946c720d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:58.920884 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8d522-0d16-42d8-b8f7-bb946c720d89 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:58.924942 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c5a36-9ec2-461b-a7cb-a628e518f142 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fbaeb7583a456d81cf79f6b101fb1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:58.938749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c5a36-9ec2-461b-a7cb-a628e518f142 service placement] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:58.953523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c5a36-9ec2-461b-a7cb-a628e518f142 service placement] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:58.953861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c5a36-9ec2-461b-a7cb-a628e518f142 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:58.957533 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c90c5a36-9ec2-461b-a7cb-a628e518f142 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:58.997180 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c5a36-9ec2-461b-a7cb-a628e518f142 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:58.998922 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c5a36-9ec2-461b-a7cb-a628e518f142 service placement] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:59.047813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1362/2754] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:52:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:53:03.730689 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802d925-0d05-4e35-92b2-6b00742f2219 service swift] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:03.799965 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802d925-0d05-4e35-92b2-6b00742f2219 service swift] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:03.805784 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4baa0-e6a3-4685-8869-55504e32160c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:03.806829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4baa0-e6a3-4685-8869-55504e32160c service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:03.807191 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4baa0-e6a3-4685-8869-55504e32160c service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:03.807573 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4baa0-e6a3-4685-8869-55504e32160c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:03.818913 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-23b4baa0-e6a3-4685-8869-55504e32160c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:03.929951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4baa0-e6a3-4685-8869-55504e32160c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:03.935144 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4baa0-e6a3-4685-8869-55504e32160c service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:03.960520 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1393/2755] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:53:04.568780 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c5a36-9ec2-461b-a7cb-a628e518f142 service placement] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:04.571000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c90c5a36-9ec2-461b-a7cb-a628e518f142 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:04.639724 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c5a36-9ec2-461b-a7cb-a628e518f142 service placement] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:04.648030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc4849-fc76-423e-a083-71145ef0b927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:04.649228 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc4849-fc76-423e-a083-71145ef0b927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:04.649519 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc4849-fc76-423e-a083-71145ef0b927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:04.649798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc4849-fc76-423e-a083-71145ef0b927 admin admin] PATH_INFO: `/v3/users/ba0229c6732541b4a6dda9040c038686` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:04.651386 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc4849-fc76-423e-a083-71145ef0b927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba0229c6732541b4a6dda9040c038686)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:04.654038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc4849-fc76-423e-a083-71145ef0b927 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:04.847327 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-82fc4849-fc76-423e-a083-71145ef0b927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba0229c6732541b4a6dda9040c038686'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:04.848017 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-82fc4849-fc76-423e-a083-71145ef0b927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba0229c6732541b4a6dda9040c038686'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:04.848631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-82fc4849-fc76-423e-a083-71145ef0b927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba0229c6732541b4a6dda9040c038686'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:04.857769 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-82fc4849-fc76-423e-a083-71145ef0b927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba0229c6732541b4a6dda9040c038686'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:04.882996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-82fc4849-fc76-423e-a083-71145ef0b927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba0229c6732541b4a6dda9040c038686'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:04.973626 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1363/2756] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:53:04 2022] DELETE /identity/v3/users/ba0229c6732541b4a6dda9040c038686 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:05.775068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4baa0-e6a3-4685-8869-55504e32160c service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:05.786731 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4baa0-e6a3-4685-8869-55504e32160c service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:05.871660 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:05.873084 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:05.873217 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:05.873453 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54 admin admin] PATH_INFO: `/v3/projects/6263581d20ea47bcae22d736f7934e05` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:05.874693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6263581d20ea47bcae22d736f7934e05)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:05.876920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:06.021864 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:06.037404 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6263581d20ea47bcae22d736f7934e05', 'request_id': 'req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:06.047459 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6263581d20ea47bcae22d736f7934e05', 'request_id': 'req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:06.122099 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6263581d20ea47bcae22d736f7934e05', 'request_id': 'req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:06.135035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54 admin admin] The token cache is being invalidate because project 6263581d20ea47bcae22d736f7934e05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:53:06.136297 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:06.137243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:06.139647 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1394/2757] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:53:05 2022] DELETE /identity/v3/projects/6263581d20ea47bcae22d736f7934e05 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:06.143793 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc4849-fc76-423e-a083-71145ef0b927 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:06.145984 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-82fc4849-fc76-423e-a083-71145ef0b927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:06.184978 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc4849-fc76-423e-a083-71145ef0b927 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:06.213967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:06.317380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6e89f-4f11-4dfc-b8b4-0f6a78d77d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:06.317380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6e89f-4f11-4dfc-b8b4-0f6a78d77d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:06.317380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6e89f-4f11-4dfc-b8b4-0f6a78d77d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:06.317380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6e89f-4f11-4dfc-b8b4-0f6a78d77d85 admin admin] PATH_INFO: `/v3/users/0a10eef43ad9475e8156c62932380057` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:06.318032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6e89f-4f11-4dfc-b8b4-0f6a78d77d85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a10eef43ad9475e8156c62932380057)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:06.318032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6e89f-4f11-4dfc-b8b4-0f6a78d77d85 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:06.803927 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0eb6e89f-4f11-4dfc-b8b4-0f6a78d77d85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a10eef43ad9475e8156c62932380057'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:06.807921 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0eb6e89f-4f11-4dfc-b8b4-0f6a78d77d85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a10eef43ad9475e8156c62932380057'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:06.808192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0eb6e89f-4f11-4dfc-b8b4-0f6a78d77d85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a10eef43ad9475e8156c62932380057'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:06.824912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0eb6e89f-4f11-4dfc-b8b4-0f6a78d77d85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a10eef43ad9475e8156c62932380057'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:06.845938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-0eb6e89f-4f11-4dfc-b8b4-0f6a78d77d85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a10eef43ad9475e8156c62932380057'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:06.937784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1364/2758] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:53:06 2022] DELETE /identity/v3/users/0a10eef43ad9475e8156c62932380057 => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:07.328219 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:07.330817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:07.377534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d73caf-8dd4-4c0d-8c54-f665f36dbf54 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:07.470758 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461bc9b-3b20-47ef-91d6-6583421452d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:07.471693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461bc9b-3b20-47ef-91d6-6583421452d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:07.471940 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461bc9b-3b20-47ef-91d6-6583421452d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:07.472234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461bc9b-3b20-47ef-91d6-6583421452d2 admin admin] PATH_INFO: `/v3/projects/561f63bffda54e37a768740ecda8dd52` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:07.473490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461bc9b-3b20-47ef-91d6-6583421452d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=561f63bffda54e37a768740ecda8dd52)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:07.476120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461bc9b-3b20-47ef-91d6-6583421452d2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:07.614812 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6461bc9b-3b20-47ef-91d6-6583421452d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '561f63bffda54e37a768740ecda8dd52', 'request_id': 'req-6461bc9b-3b20-47ef-91d6-6583421452d2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:07.625318 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6461bc9b-3b20-47ef-91d6-6583421452d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '561f63bffda54e37a768740ecda8dd52', 'request_id': 'req-6461bc9b-3b20-47ef-91d6-6583421452d2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:07.726363 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6461bc9b-3b20-47ef-91d6-6583421452d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '561f63bffda54e37a768740ecda8dd52', 'request_id': 'req-6461bc9b-3b20-47ef-91d6-6583421452d2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:07.754579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6461bc9b-3b20-47ef-91d6-6583421452d2 admin admin] The token cache is being invalidate because project 561f63bffda54e37a768740ecda8dd52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:53:07.755010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6461bc9b-3b20-47ef-91d6-6583421452d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:07.757425 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6461bc9b-3b20-47ef-91d6-6583421452d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:07.760029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1395/2759] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:53:07 2022] DELETE /identity/v3/projects/561f63bffda54e37a768740ecda8dd52 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:08.852218 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6e89f-4f11-4dfc-b8b4-0f6a78d77d85 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:08.853872 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0eb6e89f-4f11-4dfc-b8b4-0f6a78d77d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:08.870861 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6e89f-4f11-4dfc-b8b4-0f6a78d77d85 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:08.950195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb845247-588d-4671-b226-b507dcf4ac16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:08.951525 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb845247-588d-4671-b226-b507dcf4ac16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:08.952057 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb845247-588d-4671-b226-b507dcf4ac16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:08.952541 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb845247-588d-4671-b226-b507dcf4ac16 admin admin] PATH_INFO: `/v3/users/7a5fc191f3464ffe83d5fde2419cb8b4` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:08.953866 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb845247-588d-4671-b226-b507dcf4ac16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a5fc191f3464ffe83d5fde2419cb8b4)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:08.955900 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb845247-588d-4671-b226-b507dcf4ac16 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:09.356030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:09.365933 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cb845247-588d-4671-b226-b507dcf4ac16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a5fc191f3464ffe83d5fde2419cb8b4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:09.366709 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cb845247-588d-4671-b226-b507dcf4ac16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a5fc191f3464ffe83d5fde2419cb8b4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:09.367331 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cb845247-588d-4671-b226-b507dcf4ac16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a5fc191f3464ffe83d5fde2419cb8b4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:09.378771 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cb845247-588d-4671-b226-b507dcf4ac16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a5fc191f3464ffe83d5fde2419cb8b4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:09.395738 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-cb845247-588d-4671-b226-b507dcf4ac16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a5fc191f3464ffe83d5fde2419cb8b4'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:09.481881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1365/2760] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:53:08 2022] DELETE /identity/v3/users/7a5fc191f3464ffe83d5fde2419cb8b4 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:09.790690 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461bc9b-3b20-47ef-91d6-6583421452d2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:09.793515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6461bc9b-3b20-47ef-91d6-6583421452d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:09.809344 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461bc9b-3b20-47ef-91d6-6583421452d2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:09.866802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:09.876495 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c4141-ebe7-41bd-8d44-8f8f30657484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:09.877419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c4141-ebe7-41bd-8d44-8f8f30657484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:09.877710 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c4141-ebe7-41bd-8d44-8f8f30657484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:09.877988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c4141-ebe7-41bd-8d44-8f8f30657484 admin admin] PATH_INFO: `/v3/projects/749e4849b23b4e42be64ef0ab0330a0b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:09.879169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c4141-ebe7-41bd-8d44-8f8f30657484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=749e4849b23b4e42be64ef0ab0330a0b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:09.881212 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c4141-ebe7-41bd-8d44-8f8f30657484 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:10.498498 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fb2c4141-ebe7-41bd-8d44-8f8f30657484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '749e4849b23b4e42be64ef0ab0330a0b', 'request_id': 'req-fb2c4141-ebe7-41bd-8d44-8f8f30657484'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:10.505792 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fb2c4141-ebe7-41bd-8d44-8f8f30657484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '749e4849b23b4e42be64ef0ab0330a0b', 'request_id': 'req-fb2c4141-ebe7-41bd-8d44-8f8f30657484'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:10.582974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fb2c4141-ebe7-41bd-8d44-8f8f30657484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '749e4849b23b4e42be64ef0ab0330a0b', 'request_id': 'req-fb2c4141-ebe7-41bd-8d44-8f8f30657484'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:10.597948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fb2c4141-ebe7-41bd-8d44-8f8f30657484 admin admin] The token cache is being invalidate because project 749e4849b23b4e42be64ef0ab0330a0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:53:10.598330 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fb2c4141-ebe7-41bd-8d44-8f8f30657484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:10.599953 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fb2c4141-ebe7-41bd-8d44-8f8f30657484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:10.601834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1396/2761] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:53:09 2022] DELETE /identity/v3/projects/749e4849b23b4e42be64ef0ab0330a0b => generated 0 bytes in 812 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:10.607182 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9c7e6-7a43-421c-a1e3-5546e192de60 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:10.607683 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9c7e6-7a43-421c-a1e3-5546e192de60 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:10.607820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9c7e6-7a43-421c-a1e3-5546e192de60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:10.675759 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f5c9c7e6-7a43-421c-a1e3-5546e192de60 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:10.676994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f5c9c7e6-7a43-421c-a1e3-5546e192de60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:10.749442 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1366/2762] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:10.757013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e1797-e494-4a97-8b86-ce769a03a8e7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:10.757339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e1797-e494-4a97-8b86-ce769a03a8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:10.757623 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e1797-e494-4a97-8b86-ce769a03a8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:10.780531 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-255e1797-e494-4a97-8b86-ce769a03a8e7 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:10.781796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-255e1797-e494-4a97-8b86-ce769a03a8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:10.790549 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1397/2763] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:10.795235 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9c7e6-7a43-421c-a1e3-5546e192de60 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:10.805512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9c7e6-7a43-421c-a1e3-5546e192de60 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:10.810791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3a435-94b9-48b0-b1e6-87ea022fcb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:10.811457 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3a435-94b9-48b0-b1e6-87ea022fcb3d admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:10.811749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3a435-94b9-48b0-b1e6-87ea022fcb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:10.812042 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3a435-94b9-48b0-b1e6-87ea022fcb3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:10.812750 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3a435-94b9-48b0-b1e6-87ea022fcb3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:53:10.813079 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3a435-94b9-48b0-b1e6-87ea022fcb3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:10.814713 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3a435-94b9-48b0-b1e6-87ea022fcb3d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:10.828473 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1367/2764] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:53:10.832824 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e1797-e494-4a97-8b86-ce769a03a8e7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:10.843564 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e1797-e494-4a97-8b86-ce769a03a8e7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:10.849038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a2ec3-a947-423d-9ea2-b2ae05e741fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:10.849835 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a2ec3-a947-423d-9ea2-b2ae05e741fb admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:10.850189 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a2ec3-a947-423d-9ea2-b2ae05e741fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:10.850543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a2ec3-a947-423d-9ea2-b2ae05e741fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:10.851959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a2ec3-a947-423d-9ea2-b2ae05e741fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:10.854866 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a2ec3-a947-423d-9ea2-b2ae05e741fb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:10.948158 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1398/2765] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:53:10 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:53:10.950195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3a435-94b9-48b0-b1e6-87ea022fcb3d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:10.960511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3a435-94b9-48b0-b1e6-87ea022fcb3d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:11.017428 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40734d6-bb89-4d77-8315-4d833adf255e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:11.018029 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40734d6-bb89-4d77-8315-4d833adf255e admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:11.018316 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40734d6-bb89-4d77-8315-4d833adf255e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:11.018593 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40734d6-bb89-4d77-8315-4d833adf255e admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:11.019277 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40734d6-bb89-4d77-8315-4d833adf255e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:11.021233 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40734d6-bb89-4d77-8315-4d833adf255e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:11.160349 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:11.166516 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1368/2766] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:53:10 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:53:11.170902 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a2ec3-a947-423d-9ea2-b2ae05e741fb admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:11.181534 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a2ec3-a947-423d-9ea2-b2ae05e741fb admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:11.186980 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e15bde-19d0-45dd-9f60-2e37fa5e20c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:11.187428 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e15bde-19d0-45dd-9f60-2e37fa5e20c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:11.187693 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e15bde-19d0-45dd-9f60-2e37fa5e20c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:11.187976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e15bde-19d0-45dd-9f60-2e37fa5e20c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:11.188738 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e15bde-19d0-45dd-9f60-2e37fa5e20c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:11.190532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e15bde-19d0-45dd-9f60-2e37fa5e20c2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:11.205555 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1399/2767] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:11.206649 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40734d6-bb89-4d77-8315-4d833adf255e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:11.216822 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40734d6-bb89-4d77-8315-4d833adf255e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:11.221532 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f12ba-7583-45d1-989d-d95298a977b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:11.222196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f12ba-7583-45d1-989d-d95298a977b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:11.222487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f12ba-7583-45d1-989d-d95298a977b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:11.222768 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f12ba-7583-45d1-989d-d95298a977b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:11.223551 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f12ba-7583-45d1-989d-d95298a977b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:11.225192 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f12ba-7583-45d1-989d-d95298a977b2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:11.235335 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1369/2768] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:11.239077 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e15bde-19d0-45dd-9f60-2e37fa5e20c2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:11.249738 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e15bde-19d0-45dd-9f60-2e37fa5e20c2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:11.254245 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4db17-7a28-4083-a200-bada2202c92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:11.255122 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4db17-7a28-4083-a200-bada2202c92e admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:11.255414 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4db17-7a28-4083-a200-bada2202c92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:11.255701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4db17-7a28-4083-a200-bada2202c92e admin admin] PATH_INFO: `/v3/projects/184cfe90c8994b248a82362b7871b129/users/55b4228d0eee4d909817b89506568e60/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:11.273185 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4db17-7a28-4083-a200-bada2202c92e admin admin] RBAC: Authorizing `identity:create_grant(project_id=184cfe90c8994b248a82362b7871b129, user_id=55b4228d0eee4d909817b89506568e60, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:11.278200 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4db17-7a28-4083-a200-bada2202c92e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:11.377475 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1400/2769] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:53:11 2022] PUT /identity/v3/projects/184cfe90c8994b248a82362b7871b129/users/55b4228d0eee4d909817b89506568e60/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:11.388817 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601fd1d-4b6d-4b59-b5f4-d2852fb39431 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:11.389165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601fd1d-4b6d-4b59-b5f4-d2852fb39431 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:11.389451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601fd1d-4b6d-4b59-b5f4-d2852fb39431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:11.416499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d601fd1d-4b6d-4b59-b5f4-d2852fb39431 None None] MFA Rules not processed for user `55b4228d0eee4d909817b89506568e60`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:11.417881 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d601fd1d-4b6d-4b59-b5f4-d2852fb39431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:11.487113 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1370/2770] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:11.491810 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4db17-7a28-4083-a200-bada2202c92e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:11.554390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4db17-7a28-4083-a200-bada2202c92e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:11.559937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af1284-e2b1-48a1-b19f-10c491e6875d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:11.560601 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af1284-e2b1-48a1-b19f-10c491e6875d admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:11.560891 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af1284-e2b1-48a1-b19f-10c491e6875d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:11.561211 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af1284-e2b1-48a1-b19f-10c491e6875d admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:11.561908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af1284-e2b1-48a1-b19f-10c491e6875d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:11.563925 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af1284-e2b1-48a1-b19f-10c491e6875d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:11.579505 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5921d4d-b482-45dd-a530-94434d50f638 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:11.579798 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5921d4d-b482-45dd-a530-94434d50f638 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:11.580082 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5921d4d-b482-45dd-a530-94434d50f638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:11.597352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-f5921d4d-b482-45dd-a530-94434d50f638 None None] MFA Rules not processed for user `172c28d985cb40409d563fad4fa2ad5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:11.598581 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-f5921d4d-b482-45dd-a530-94434d50f638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:11.652804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1371/2771] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:53:11.663570 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1401/2772] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:53:11 2022] POST /identity/v3/projects => generated 381 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:53:11.669686 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5921d4d-b482-45dd-a530-94434d50f638 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:11.686395 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5921d4d-b482-45dd-a530-94434d50f638 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:11.745544 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f90c5-8d0f-4787-8a6a-9062a3c61bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:11.746514 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f90c5-8d0f-4787-8a6a-9062a3c61bca admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:11.746804 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f90c5-8d0f-4787-8a6a-9062a3c61bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:11.747117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f90c5-8d0f-4787-8a6a-9062a3c61bca admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:11.747792 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f90c5-8d0f-4787-8a6a-9062a3c61bca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:11.749805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f90c5-8d0f-4787-8a6a-9062a3c61bca admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:11.858193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1372/2773] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:53:11 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:53:11.862289 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af1284-e2b1-48a1-b19f-10c491e6875d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:11.868985 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:11.873767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af1284-e2b1-48a1-b19f-10c491e6875d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:11.878484 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8f382-d9d2-427e-beda-dc610ac1810c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:11.879227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8f382-d9d2-427e-beda-dc610ac1810c admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:11.879634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8f382-d9d2-427e-beda-dc610ac1810c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:11.879989 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8f382-d9d2-427e-beda-dc610ac1810c admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:11.880786 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8f382-d9d2-427e-beda-dc610ac1810c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:11.882617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8f382-d9d2-427e-beda-dc610ac1810c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:11.895682 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1402/2774] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:11.899459 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f90c5-8d0f-4787-8a6a-9062a3c61bca admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:11.911356 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f90c5-8d0f-4787-8a6a-9062a3c61bca admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:11.915481 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b30dc-7955-4302-a74d-26857ac4e30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:11.916400 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b30dc-7955-4302-a74d-26857ac4e30b admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:11.916710 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b30dc-7955-4302-a74d-26857ac4e30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:11.917244 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b30dc-7955-4302-a74d-26857ac4e30b admin admin] PATH_INFO: `/v3/projects/5c6b6965cced40ddac0f3bbf94a5c58c/users/de68dfa3c147462ebe28f5c5dd9863fd/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:11.923309 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8f382-d9d2-427e-beda-dc610ac1810c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:11.925336 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-08e8f382-d9d2-427e-beda-dc610ac1810c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:11.927701 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:11.935132 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b30dc-7955-4302-a74d-26857ac4e30b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6b6965cced40ddac0f3bbf94a5c58c, user_id=de68dfa3c147462ebe28f5c5dd9863fd, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:11.938155 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b30dc-7955-4302-a74d-26857ac4e30b admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:11.996997 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8f382-d9d2-427e-beda-dc610ac1810c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:12.000679 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c290a85-2d60-4b4f-812d-569eed767de6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:12.001372 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c290a85-2d60-4b4f-812d-569eed767de6 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:12.011013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c290a85-2d60-4b4f-812d-569eed767de6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:12.011013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c290a85-2d60-4b4f-812d-569eed767de6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:12.011013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8c290a85-2d60-4b4f-812d-569eed767de6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:12.015132 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c290a85-2d60-4b4f-812d-569eed767de6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:12.016617 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c290a85-2d60-4b4f-812d-569eed767de6 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:12.024755 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1403/2775] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:53:12.033490 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1373/2776] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:53:11 2022] PUT /identity/v3/projects/5c6b6965cced40ddac0f3bbf94a5c58c/users/de68dfa3c147462ebe28f5c5dd9863fd/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:12.037419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c290a85-2d60-4b4f-812d-569eed767de6 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:12.048911 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c290a85-2d60-4b4f-812d-569eed767de6 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:12.106319 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c625bd3-c3f1-4059-9c4e-c827ef9115f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:12.106924 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c625bd3-c3f1-4059-9c4e-c827ef9115f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:12.107221 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c625bd3-c3f1-4059-9c4e-c827ef9115f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:12.107523 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c625bd3-c3f1-4059-9c4e-c827ef9115f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:12.108251 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c625bd3-c3f1-4059-9c4e-c827ef9115f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:12.109897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c625bd3-c3f1-4059-9c4e-c827ef9115f5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:12.120306 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1404/2777] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:12.124536 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b30dc-7955-4302-a74d-26857ac4e30b admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:12.134862 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b30dc-7955-4302-a74d-26857ac4e30b admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:12.140663 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85e2c0-4e1b-47ae-8725-714ddb975924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:12.141600 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85e2c0-4e1b-47ae-8725-714ddb975924 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:12.141888 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85e2c0-4e1b-47ae-8725-714ddb975924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:12.142168 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85e2c0-4e1b-47ae-8725-714ddb975924 admin admin] PATH_INFO: `/v3/projects/5c6b6965cced40ddac0f3bbf94a5c58c/users/de68dfa3c147462ebe28f5c5dd9863fd/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:12.144142 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85e2c0-4e1b-47ae-8725-714ddb975924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6b6965cced40ddac0f3bbf94a5c58c, user_id=de68dfa3c147462ebe28f5c5dd9863fd, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:12.146934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85e2c0-4e1b-47ae-8725-714ddb975924 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:12.158403 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1374/2778] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:53:12 2022] PUT /identity/v3/projects/5c6b6965cced40ddac0f3bbf94a5c58c/users/de68dfa3c147462ebe28f5c5dd9863fd/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:12.168943 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0933949-730f-4a86-9584-b7b8c171a524 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:12.169374 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0933949-730f-4a86-9584-b7b8c171a524 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:12.169665 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0933949-730f-4a86-9584-b7b8c171a524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:12.199575 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b0933949-730f-4a86-9584-b7b8c171a524 None None] MFA Rules not processed for user `de68dfa3c147462ebe28f5c5dd9863fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:12.200422 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b0933949-730f-4a86-9584-b7b8c171a524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:12.273308 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1405/2779] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:53:12.285280 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85e2c0-4e1b-47ae-8725-714ddb975924 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:12.287006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ba85e2c0-4e1b-47ae-8725-714ddb975924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:12.345673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85e2c0-4e1b-47ae-8725-714ddb975924 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:12.349451 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b12791-64f6-468c-8907-bfe496b27e10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:12.350095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b12791-64f6-468c-8907-bfe496b27e10 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:12.350385 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b12791-64f6-468c-8907-bfe496b27e10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:12.350665 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b12791-64f6-468c-8907-bfe496b27e10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:12.367627 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b12791-64f6-468c-8907-bfe496b27e10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:12.369002 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b12791-64f6-468c-8907-bfe496b27e10 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:12.380370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1375/2780] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:53:12.410590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0933949-730f-4a86-9584-b7b8c171a524 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:12.412122 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b0933949-730f-4a86-9584-b7b8c171a524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:12.422638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:12.492105 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0933949-730f-4a86-9584-b7b8c171a524 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:12.498421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9394d70-a94b-4c54-ae4b-1ba3ee7e06d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fbaeb7583a456d81cf79f6b101fb1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:12.499164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9394d70-a94b-4c54-ae4b-1ba3ee7e06d4 service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:12.499514 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9394d70-a94b-4c54-ae4b-1ba3ee7e06d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:12.499856 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9394d70-a94b-4c54-ae4b-1ba3ee7e06d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:12.501859 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e9394d70-a94b-4c54-ae4b-1ba3ee7e06d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:12.514333 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9394d70-a94b-4c54-ae4b-1ba3ee7e06d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:12.515912 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9394d70-a94b-4c54-ae4b-1ba3ee7e06d4 service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:12.527055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1406/2781] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:53:12.591618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b12791-64f6-468c-8907-bfe496b27e10 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:12.601189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b12791-64f6-468c-8907-bfe496b27e10 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:12.605370 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64805151-13b2-4d07-8613-f14423401341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:12.605929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64805151-13b2-4d07-8613-f14423401341 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:12.606226 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64805151-13b2-4d07-8613-f14423401341 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:12.606633 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64805151-13b2-4d07-8613-f14423401341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:12.668148 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64805151-13b2-4d07-8613-f14423401341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:12.669693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64805151-13b2-4d07-8613-f14423401341 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:12.678937 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1376/2782] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:53:15.459732 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970019d-d7cd-48fc-a48a-c8dbf73f81c4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:15.460231 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970019d-d7cd-48fc-a48a-c8dbf73f81c4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:15.460361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970019d-d7cd-48fc-a48a-c8dbf73f81c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:15.478587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0970019d-d7cd-48fc-a48a-c8dbf73f81c4 None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:15.479807 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0970019d-d7cd-48fc-a48a-c8dbf73f81c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:15.540019 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1407/2783] 10.222.0.64 () {58 vars in 1009 bytes} [Mon May 23 15:53:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:15.552600 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64805151-13b2-4d07-8613-f14423401341 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:15.554254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-64805151-13b2-4d07-8613-f14423401341 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:15.663117 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64805151-13b2-4d07-8613-f14423401341 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:15.668710 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c0ad5-cc60-4792-b7ac-d4ac2c56c30c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:15.671854 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c0ad5-cc60-4792-b7ac-d4ac2c56c30c service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:15.671854 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c0ad5-cc60-4792-b7ac-d4ac2c56c30c service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:15.671854 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c0ad5-cc60-4792-b7ac-d4ac2c56c30c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:15.679174 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49c0ad5-cc60-4792-b7ac-d4ac2c56c30c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:15.680862 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49c0ad5-cc60-4792-b7ac-d4ac2c56c30c service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:15.689892 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1377/2784] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:53:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:53:25.533359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=54 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:25.686322 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=96,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:25.928013 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970019d-d7cd-48fc-a48a-c8dbf73f81c4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:25.991628 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970019d-d7cd-48fc-a48a-c8dbf73f81c4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:25.996763 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e7e6f-8310-4217-a44c-4a9afc65d591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:25.997668 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e7e6f-8310-4217-a44c-4a9afc65d591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:25.998064 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e7e6f-8310-4217-a44c-4a9afc65d591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:25.998350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e7e6f-8310-4217-a44c-4a9afc65d591 admin admin] PATH_INFO: `/v3/users/55b4228d0eee4d909817b89506568e60` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:25.999590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e7e6f-8310-4217-a44c-4a9afc65d591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55b4228d0eee4d909817b89506568e60)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:26.001555 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e7e6f-8310-4217-a44c-4a9afc65d591 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:26.184796 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f53e7e6f-8310-4217-a44c-4a9afc65d591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55b4228d0eee4d909817b89506568e60'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:26.196354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f53e7e6f-8310-4217-a44c-4a9afc65d591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55b4228d0eee4d909817b89506568e60'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:26.256492 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f53e7e6f-8310-4217-a44c-4a9afc65d591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55b4228d0eee4d909817b89506568e60'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:26.257131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f53e7e6f-8310-4217-a44c-4a9afc65d591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55b4228d0eee4d909817b89506568e60'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:26.264308 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f53e7e6f-8310-4217-a44c-4a9afc65d591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55b4228d0eee4d909817b89506568e60'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:26.270334 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1408/2785] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:53:25 2022] DELETE /identity/v3/users/55b4228d0eee4d909817b89506568e60 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:26.274884 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74591393-b916-4c3e-80af-13c1f8485e1b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:26.275707 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74591393-b916-4c3e-80af-13c1f8485e1b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:26.276020 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74591393-b916-4c3e-80af-13c1f8485e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:26.294060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-74591393-b916-4c3e-80af-13c1f8485e1b None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:26.295305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-74591393-b916-4c3e-80af-13c1f8485e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:26.354401 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1378/2786] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:26.363859 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e7e6f-8310-4217-a44c-4a9afc65d591 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:26.365285 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f53e7e6f-8310-4217-a44c-4a9afc65d591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:26.422899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e7e6f-8310-4217-a44c-4a9afc65d591 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:26.426903 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ad65f-1f00-4359-98e4-6442911e6cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:26.427542 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ad65f-1f00-4359-98e4-6442911e6cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:26.427823 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ad65f-1f00-4359-98e4-6442911e6cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:26.428099 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ad65f-1f00-4359-98e4-6442911e6cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:26.437592 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ad65f-1f00-4359-98e4-6442911e6cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:26.438935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ad65f-1f00-4359-98e4-6442911e6cd9 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:26.449282 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1409/2787] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:53:26.921570 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74591393-b916-4c3e-80af-13c1f8485e1b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:26.923174 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-74591393-b916-4c3e-80af-13c1f8485e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:26.949565 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74591393-b916-4c3e-80af-13c1f8485e1b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:26.954257 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37e23a-30d0-4f2d-87c1-de59127a4945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:26.959849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37e23a-30d0-4f2d-87c1-de59127a4945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:26.959849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37e23a-30d0-4f2d-87c1-de59127a4945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:26.959849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37e23a-30d0-4f2d-87c1-de59127a4945 admin admin] PATH_INFO: `/v3/projects/184cfe90c8994b248a82362b7871b129` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:26.959849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37e23a-30d0-4f2d-87c1-de59127a4945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=184cfe90c8994b248a82362b7871b129)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:26.959849 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37e23a-30d0-4f2d-87c1-de59127a4945 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:27.080793 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fa37e23a-30d0-4f2d-87c1-de59127a4945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '184cfe90c8994b248a82362b7871b129', 'request_id': 'req-fa37e23a-30d0-4f2d-87c1-de59127a4945'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:27.087189 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fa37e23a-30d0-4f2d-87c1-de59127a4945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '184cfe90c8994b248a82362b7871b129', 'request_id': 'req-fa37e23a-30d0-4f2d-87c1-de59127a4945'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:27.093642 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fa37e23a-30d0-4f2d-87c1-de59127a4945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '184cfe90c8994b248a82362b7871b129', 'request_id': 'req-fa37e23a-30d0-4f2d-87c1-de59127a4945'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:27.159224 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fa37e23a-30d0-4f2d-87c1-de59127a4945 admin admin] The token cache is being invalidate because project 184cfe90c8994b248a82362b7871b129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:53:27.159546 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fa37e23a-30d0-4f2d-87c1-de59127a4945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:27.160205 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-fa37e23a-30d0-4f2d-87c1-de59127a4945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:27.161827 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1379/2788] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:53:26 2022] DELETE /identity/v3/projects/184cfe90c8994b248a82362b7871b129 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:27.166020 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ad65f-1f00-4359-98e4-6442911e6cd9 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:27.176168 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ad65f-1f00-4359-98e4-6442911e6cd9 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:27.194954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:27.239434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd1a39-d94b-43ae-ad42-5686326dd338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:27.239434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd1a39-d94b-43ae-ad42-5686326dd338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:27.239434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd1a39-d94b-43ae-ad42-5686326dd338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:27.239434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd1a39-d94b-43ae-ad42-5686326dd338 admin admin] PATH_INFO: `/v3/users/de68dfa3c147462ebe28f5c5dd9863fd` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:27.239990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd1a39-d94b-43ae-ad42-5686326dd338 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de68dfa3c147462ebe28f5c5dd9863fd)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:27.239990 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd1a39-d94b-43ae-ad42-5686326dd338 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:27.406717 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b8fd1a39-d94b-43ae-ad42-5686326dd338 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de68dfa3c147462ebe28f5c5dd9863fd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:27.419353 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b8fd1a39-d94b-43ae-ad42-5686326dd338 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de68dfa3c147462ebe28f5c5dd9863fd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:27.502544 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b8fd1a39-d94b-43ae-ad42-5686326dd338 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de68dfa3c147462ebe28f5c5dd9863fd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:27.503091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b8fd1a39-d94b-43ae-ad42-5686326dd338 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de68dfa3c147462ebe28f5c5dd9863fd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:27.510295 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b8fd1a39-d94b-43ae-ad42-5686326dd338 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de68dfa3c147462ebe28f5c5dd9863fd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:27.517284 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1410/2789] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:53:27 2022] DELETE /identity/v3/users/de68dfa3c147462ebe28f5c5dd9863fd => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:27.937979 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37e23a-30d0-4f2d-87c1-de59127a4945 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:27.939462 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-fa37e23a-30d0-4f2d-87c1-de59127a4945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:27.953240 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37e23a-30d0-4f2d-87c1-de59127a4945 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:28.016305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e15b5-fc6f-47b5-b3a2-a4ee179ae825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:28.017236 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e15b5-fc6f-47b5-b3a2-a4ee179ae825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:28.017528 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e15b5-fc6f-47b5-b3a2-a4ee179ae825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:28.017815 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e15b5-fc6f-47b5-b3a2-a4ee179ae825 admin admin] PATH_INFO: `/v3/projects/5c6b6965cced40ddac0f3bbf94a5c58c` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:28.019033 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e15b5-fc6f-47b5-b3a2-a4ee179ae825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c6b6965cced40ddac0f3bbf94a5c58c)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:28.021009 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e15b5-fc6f-47b5-b3a2-a4ee179ae825 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:28.102524 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:28.118739 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e22e15b5-fc6f-47b5-b3a2-a4ee179ae825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c6b6965cced40ddac0f3bbf94a5c58c', 'request_id': 'req-e22e15b5-fc6f-47b5-b3a2-a4ee179ae825'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:28.124794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e22e15b5-fc6f-47b5-b3a2-a4ee179ae825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c6b6965cced40ddac0f3bbf94a5c58c', 'request_id': 'req-e22e15b5-fc6f-47b5-b3a2-a4ee179ae825'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:28.130871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e22e15b5-fc6f-47b5-b3a2-a4ee179ae825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c6b6965cced40ddac0f3bbf94a5c58c', 'request_id': 'req-e22e15b5-fc6f-47b5-b3a2-a4ee179ae825'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:28.209049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e22e15b5-fc6f-47b5-b3a2-a4ee179ae825 admin admin] The token cache is being invalidate because project 5c6b6965cced40ddac0f3bbf94a5c58c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:53:28.209371 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e22e15b5-fc6f-47b5-b3a2-a4ee179ae825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:28.210005 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-e22e15b5-fc6f-47b5-b3a2-a4ee179ae825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:28.211623 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1380/2790] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:53:27 2022] DELETE /identity/v3/projects/5c6b6965cced40ddac0f3bbf94a5c58c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:28.216603 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609cbeb-0287-430b-bf89-3233ee2da9d1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:28.217121 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609cbeb-0287-430b-bf89-3233ee2da9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:28.217488 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609cbeb-0287-430b-bf89-3233ee2da9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:28.235385 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4609cbeb-0287-430b-bf89-3233ee2da9d1 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:28.236587 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4609cbeb-0287-430b-bf89-3233ee2da9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:28.297384 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1411/2791] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:28.306716 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5349393-c6e0-4779-9bb9-637302c00911 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:28.307017 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5349393-c6e0-4779-9bb9-637302c00911 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:28.307301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5349393-c6e0-4779-9bb9-637302c00911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:28.328025 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-d5349393-c6e0-4779-9bb9-637302c00911 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:28.329309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d5349393-c6e0-4779-9bb9-637302c00911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:28.337273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1381/2792] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:28.342065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609cbeb-0287-430b-bf89-3233ee2da9d1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:28.352109 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609cbeb-0287-430b-bf89-3233ee2da9d1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:28.357440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4305f-b526-4b57-af5c-63c53a6e6eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:28.358099 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4305f-b526-4b57-af5c-63c53a6e6eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:28.358391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4305f-b526-4b57-af5c-63c53a6e6eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:28.358674 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4305f-b526-4b57-af5c-63c53a6e6eb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:28.359402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4305f-b526-4b57-af5c-63c53a6e6eb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:53:28.359714 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4305f-b526-4b57-af5c-63c53a6e6eb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:28.361346 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4305f-b526-4b57-af5c-63c53a6e6eb4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:28.374206 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1412/2793] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:53:28.378463 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5349393-c6e0-4779-9bb9-637302c00911 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:28.388130 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5349393-c6e0-4779-9bb9-637302c00911 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:28.392783 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e0612-bc59-4bbb-94ac-47db7538acc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:28.393541 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e0612-bc59-4bbb-94ac-47db7538acc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:28.393828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e0612-bc59-4bbb-94ac-47db7538acc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:28.394106 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e0612-bc59-4bbb-94ac-47db7538acc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:28.394799 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e0612-bc59-4bbb-94ac-47db7538acc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:28.396690 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e0612-bc59-4bbb-94ac-47db7538acc4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:28.475095 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1382/2794] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:53:28 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:53:28.479146 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4305f-b526-4b57-af5c-63c53a6e6eb4 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:28.490163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4305f-b526-4b57-af5c-63c53a6e6eb4 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:28.516920 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:28.547772 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3b065-59d0-431b-abb8-4a95e28da9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:28.548412 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3b065-59d0-431b-abb8-4a95e28da9fc admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:28.548705 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3b065-59d0-431b-abb8-4a95e28da9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:28.548991 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3b065-59d0-431b-abb8-4a95e28da9fc admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:28.549983 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd3b065-59d0-431b-abb8-4a95e28da9fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:28.551951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd3b065-59d0-431b-abb8-4a95e28da9fc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:28.691755 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1413/2795] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:53:28 2022] POST /identity/v3/users => generated 291 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:53:28.695744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e0612-bc59-4bbb-94ac-47db7538acc4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:28.718032 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e0612-bc59-4bbb-94ac-47db7538acc4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:28.722784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c21f58-a47e-47ad-9c90-92f0d7a2c7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:28.723453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c21f58-a47e-47ad-9c90-92f0d7a2c7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:28.723766 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c21f58-a47e-47ad-9c90-92f0d7a2c7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:28.724045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c21f58-a47e-47ad-9c90-92f0d7a2c7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:28.724901 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c21f58-a47e-47ad-9c90-92f0d7a2c7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:28.726614 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c21f58-a47e-47ad-9c90-92f0d7a2c7e4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:28.740992 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1383/2796] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:28.742393 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3b065-59d0-431b-abb8-4a95e28da9fc admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:28.752887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3b065-59d0-431b-abb8-4a95e28da9fc admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:28.757737 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf6ae1-19f3-4d8a-ab2a-87a9adc70527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:28.758446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf6ae1-19f3-4d8a-ab2a-87a9adc70527 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:28.758736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf6ae1-19f3-4d8a-ab2a-87a9adc70527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:28.759043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf6ae1-19f3-4d8a-ab2a-87a9adc70527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:28.759767 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf6ae1-19f3-4d8a-ab2a-87a9adc70527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:28.761897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf6ae1-19f3-4d8a-ab2a-87a9adc70527 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:28.772040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1414/2797] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:28.776181 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c21f58-a47e-47ad-9c90-92f0d7a2c7e4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:28.786996 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c21f58-a47e-47ad-9c90-92f0d7a2c7e4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:28.791905 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c85c0-d1b3-44ca-9ad4-4ac125d27ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:28.793348 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c85c0-d1b3-44ca-9ad4-4ac125d27ca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:28.793744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c85c0-d1b3-44ca-9ad4-4ac125d27ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:28.794101 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c85c0-d1b3-44ca-9ad4-4ac125d27ca7 admin admin] PATH_INFO: `/v3/projects/60721ae81b9e4141bc7275fe248235fb/users/4d11c23f121f474091d7ae232505ece4/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:28.809361 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c85c0-d1b3-44ca-9ad4-4ac125d27ca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60721ae81b9e4141bc7275fe248235fb, user_id=4d11c23f121f474091d7ae232505ece4, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:28.812144 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c85c0-d1b3-44ca-9ad4-4ac125d27ca7 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:28.835638 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf6ae1-19f3-4d8a-ab2a-87a9adc70527 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:28.837418 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a2bf6ae1-19f3-4d8a-ab2a-87a9adc70527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:28.910281 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf6ae1-19f3-4d8a-ab2a-87a9adc70527 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:28.917449 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18966d6-d63e-4f32-9896-4bf845d5939b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da96bed73794fee8f719ec904936164', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:28.918114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18966d6-d63e-4f32-9896-4bf845d5939b service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:28.918409 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18966d6-d63e-4f32-9896-4bf845d5939b service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:28.918800 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18966d6-d63e-4f32-9896-4bf845d5939b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:28.920703 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e18966d6-d63e-4f32-9896-4bf845d5939b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:28.929868 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1384/2798] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:53:28 2022] PUT /identity/v3/projects/60721ae81b9e4141bc7275fe248235fb/users/4d11c23f121f474091d7ae232505ece4/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:28.932961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11dfc3-4321-4cd2-8067-c847146cc667 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:28.932961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11dfc3-4321-4cd2-8067-c847146cc667 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:28.934087 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18966d6-d63e-4f32-9896-4bf845d5939b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:28.934392 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11dfc3-4321-4cd2-8067-c847146cc667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:28.935718 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18966d6-d63e-4f32-9896-4bf845d5939b service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:28.947343 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1415/2799] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:53:28.966810 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-6b11dfc3-4321-4cd2-8067-c847146cc667 None None] MFA Rules not processed for user `4d11c23f121f474091d7ae232505ece4`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:28.968582 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-6b11dfc3-4321-4cd2-8067-c847146cc667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:29.038226 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1385/2800] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:29.043983 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18966d6-d63e-4f32-9896-4bf845d5939b service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:29.054675 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18966d6-d63e-4f32-9896-4bf845d5939b service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:29.115630 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f07b5-1855-4800-82af-2f240e97db02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:29.116319 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f07b5-1855-4800-82af-2f240e97db02 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:29.116639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f07b5-1855-4800-82af-2f240e97db02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:29.116919 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f07b5-1855-4800-82af-2f240e97db02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:29.117685 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f07b5-1855-4800-82af-2f240e97db02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:29.126267 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f07b5-1855-4800-82af-2f240e97db02 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:29.246574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1416/2801] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:53:29 2022] POST /identity/v3/projects => generated 379 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:53:29.252317 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11dfc3-4321-4cd2-8067-c847146cc667 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:29.263120 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11dfc3-4321-4cd2-8067-c847146cc667 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:29.302238 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:29.323890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f454947-7aa2-49df-9dcc-ff7b025ba138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:29.323890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f454947-7aa2-49df-9dcc-ff7b025ba138 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:29.323890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f454947-7aa2-49df-9dcc-ff7b025ba138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:29.323890 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f454947-7aa2-49df-9dcc-ff7b025ba138 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:29.324384 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f454947-7aa2-49df-9dcc-ff7b025ba138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:29.324499 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f454947-7aa2-49df-9dcc-ff7b025ba138 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:29.427398 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1386/2802] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:53:29 2022] POST /identity/v3/users => generated 296 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:53:29.431394 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f07b5-1855-4800-82af-2f240e97db02 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:29.442515 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f07b5-1855-4800-82af-2f240e97db02 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:29.447443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0812a-5723-4217-85d1-00d8ef3178fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:29.448140 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0812a-5723-4217-85d1-00d8ef3178fe admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:29.448431 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0812a-5723-4217-85d1-00d8ef3178fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:29.448777 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0812a-5723-4217-85d1-00d8ef3178fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:29.450360 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0812a-5723-4217-85d1-00d8ef3178fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:29.452007 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0812a-5723-4217-85d1-00d8ef3178fe admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:29.462646 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1417/2803] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:29.468147 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f454947-7aa2-49df-9dcc-ff7b025ba138 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:29.478222 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f454947-7aa2-49df-9dcc-ff7b025ba138 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:29.483278 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c8ea4-75e1-45ac-acb9-0f7e710af005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:29.484225 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c8ea4-75e1-45ac-acb9-0f7e710af005 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:29.484711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c8ea4-75e1-45ac-acb9-0f7e710af005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:29.485328 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c8ea4-75e1-45ac-acb9-0f7e710af005 admin admin] PATH_INFO: `/v3/projects/d35a48109c5e4b90a2a53cf0e21ececb/users/779f44712ddf41f5bbf518c6b53c3feb/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:29.500564 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c8ea4-75e1-45ac-acb9-0f7e710af005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d35a48109c5e4b90a2a53cf0e21ececb, user_id=779f44712ddf41f5bbf518c6b53c3feb, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:29.503249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c8ea4-75e1-45ac-acb9-0f7e710af005 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:29.570867 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1387/2804] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:53:29 2022] PUT /identity/v3/projects/d35a48109c5e4b90a2a53cf0e21ececb/users/779f44712ddf41f5bbf518c6b53c3feb/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:29.580012 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0812a-5723-4217-85d1-00d8ef3178fe admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:29.584672 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0812a-5723-4217-85d1-00d8ef3178fe admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:29.588463 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:29.641076 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc50a7-85fb-4c9c-9136-3e3aa26aae62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:29.641761 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc50a7-85fb-4c9c-9136-3e3aa26aae62 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:29.642045 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc50a7-85fb-4c9c-9136-3e3aa26aae62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:29.642322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc50a7-85fb-4c9c-9136-3e3aa26aae62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:29.643035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc50a7-85fb-4c9c-9136-3e3aa26aae62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:29.644612 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc50a7-85fb-4c9c-9136-3e3aa26aae62 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:29.663106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1418/2805] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:29.668559 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c8ea4-75e1-45ac-acb9-0f7e710af005 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:29.679487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c8ea4-75e1-45ac-acb9-0f7e710af005 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:29.685011 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba07ff-e13c-4bd4-b1b1-969c8b26da5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:29.685930 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba07ff-e13c-4bd4-b1b1-969c8b26da5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:29.686212 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba07ff-e13c-4bd4-b1b1-969c8b26da5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:29.686489 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba07ff-e13c-4bd4-b1b1-969c8b26da5e admin admin] PATH_INFO: `/v3/projects/d35a48109c5e4b90a2a53cf0e21ececb/users/779f44712ddf41f5bbf518c6b53c3feb/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:29.688449 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba07ff-e13c-4bd4-b1b1-969c8b26da5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d35a48109c5e4b90a2a53cf0e21ececb, user_id=779f44712ddf41f5bbf518c6b53c3feb, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:29.691167 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba07ff-e13c-4bd4-b1b1-969c8b26da5e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:29.701831 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1388/2806] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:53:29 2022] PUT /identity/v3/projects/d35a48109c5e4b90a2a53cf0e21ececb/users/779f44712ddf41f5bbf518c6b53c3feb/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:29.711935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e212e-1c87-4ae3-be54-ab39b0bf5e67 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:29.712263 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e212e-1c87-4ae3-be54-ab39b0bf5e67 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:29.712551 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e212e-1c87-4ae3-be54-ab39b0bf5e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:29.739208 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-468e212e-1c87-4ae3-be54-ab39b0bf5e67 None None] MFA Rules not processed for user `779f44712ddf41f5bbf518c6b53c3feb`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:29.740402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-468e212e-1c87-4ae3-be54-ab39b0bf5e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:29.807447 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1419/2807] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:29 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:53:29.826608 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba07ff-e13c-4bd4-b1b1-969c8b26da5e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:29.828305 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a2ba07ff-e13c-4bd4-b1b1-969c8b26da5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:29.885834 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba07ff-e13c-4bd4-b1b1-969c8b26da5e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:29.889373 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36c78f-44c7-448b-8a87-4cfaaa1dbda8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:29.890018 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36c78f-44c7-448b-8a87-4cfaaa1dbda8 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:29.890320 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36c78f-44c7-448b-8a87-4cfaaa1dbda8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:29.890589 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36c78f-44c7-448b-8a87-4cfaaa1dbda8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:29.899719 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36c78f-44c7-448b-8a87-4cfaaa1dbda8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:29.906352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36c78f-44c7-448b-8a87-4cfaaa1dbda8 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:29.911476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1389/2808] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:53:31.197915 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e212e-1c87-4ae3-be54-ab39b0bf5e67 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:31.207614 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e212e-1c87-4ae3-be54-ab39b0bf5e67 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:31.212237 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c23c87-80e9-43db-84cb-fbb31c3f3595 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '779f44712ddf41f5bbf518c6b53c3feb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd35a48109c5e4b90a2a53cf0e21ececb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:31.212899 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c23c87-80e9-43db-84cb-fbb31c3f3595 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:31.213225 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c23c87-80e9-43db-84cb-fbb31c3f3595 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:31.213508 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c23c87-80e9-43db-84cb-fbb31c3f3595 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:31.214265 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c23c87-80e9-43db-84cb-fbb31c3f3595 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:53:31.214996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c23c87-80e9-43db-84cb-fbb31c3f3595 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:31.216590 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c23c87-80e9-43db-84cb-fbb31c3f3595 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:31.228043 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1420/2809] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:53:31.233093 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36c78f-44c7-448b-8a87-4cfaaa1dbda8 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:31.242107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36c78f-44c7-448b-8a87-4cfaaa1dbda8 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:31.247256 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ca3a5-bd9f-4109-8b10-46739e41883b tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '779f44712ddf41f5bbf518c6b53c3feb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd35a48109c5e4b90a2a53cf0e21ececb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:31.247893 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ca3a5-bd9f-4109-8b10-46739e41883b tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:31.248181 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ca3a5-bd9f-4109-8b10-46739e41883b tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:31.248462 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ca3a5-bd9f-4109-8b10-46739e41883b tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:31.249187 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ca3a5-bd9f-4109-8b10-46739e41883b tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:31.251082 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ca3a5-bd9f-4109-8b10-46739e41883b tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:31.350000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1390/2810] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:53:31 2022] POST /identity/v3/projects => generated 354 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:53:31.353996 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c23c87-80e9-43db-84cb-fbb31c3f3595 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:31.364485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c23c87-80e9-43db-84cb-fbb31c3f3595 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:31.422739 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5feda8f-469b-49b9-a0a2-b13c4014838d tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '779f44712ddf41f5bbf518c6b53c3feb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd35a48109c5e4b90a2a53cf0e21ececb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:31.423091 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5feda8f-469b-49b9-a0a2-b13c4014838d tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:31.423278 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5feda8f-469b-49b9-a0a2-b13c4014838d tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:31.423563 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5feda8f-469b-49b9-a0a2-b13c4014838d tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:31.424659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5feda8f-469b-49b9-a0a2-b13c4014838d tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:53:31.424976 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5feda8f-469b-49b9-a0a2-b13c4014838d tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:31.426734 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5feda8f-469b-49b9-a0a2-b13c4014838d tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:31.439073 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1421/2811] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:53:31.476794 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ca3a5-bd9f-4109-8b10-46739e41883b tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:31.488125 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ca3a5-bd9f-4109-8b10-46739e41883b tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:31.493658 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d7a4b-23af-420c-9794-7089e647eeb0 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '779f44712ddf41f5bbf518c6b53c3feb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd35a48109c5e4b90a2a53cf0e21ececb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:31.494588 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d7a4b-23af-420c-9794-7089e647eeb0 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:31.494991 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d7a4b-23af-420c-9794-7089e647eeb0 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:31.495341 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d7a4b-23af-420c-9794-7089e647eeb0 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] PATH_INFO: `/v3/projects/8a6504de7a0e4d3ba903ad44d08cc4d6` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:31.496611 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d7a4b-23af-420c-9794-7089e647eeb0 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8a6504de7a0e4d3ba903ad44d08cc4d6)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:31.498797 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d7a4b-23af-420c-9794-7089e647eeb0 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:31.594634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6b7d7a4b-23af-420c-9794-7089e647eeb0 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a6504de7a0e4d3ba903ad44d08cc4d6', 'request_id': 'req-6b7d7a4b-23af-420c-9794-7089e647eeb0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:31.600159 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6b7d7a4b-23af-420c-9794-7089e647eeb0 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a6504de7a0e4d3ba903ad44d08cc4d6', 'request_id': 'req-6b7d7a4b-23af-420c-9794-7089e647eeb0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:31.607871 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6b7d7a4b-23af-420c-9794-7089e647eeb0 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a6504de7a0e4d3ba903ad44d08cc4d6', 'request_id': 'req-6b7d7a4b-23af-420c-9794-7089e647eeb0'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:31.611739 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:31.679279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6b7d7a4b-23af-420c-9794-7089e647eeb0 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] The token cache is being invalidate because project 8a6504de7a0e4d3ba903ad44d08cc4d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:53:31.683509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6b7d7a4b-23af-420c-9794-7089e647eeb0 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:31.683509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-6b7d7a4b-23af-420c-9794-7089e647eeb0 tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:31.683509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1391/2812] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:53:31 2022] DELETE /identity/v3/projects/8a6504de7a0e4d3ba903ad44d08cc4d6 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:32.243296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5feda8f-469b-49b9-a0a2-b13c4014838d tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:32.245259 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a5feda8f-469b-49b9-a0a2-b13c4014838d tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:32.257464 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:32.312139 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5feda8f-469b-49b9-a0a2-b13c4014838d tempest-VolumeQuotaClassesTest-305846654 tempest-VolumeQuotaClassesTest-305846654-project-admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:32.371666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dd32c-036d-46a3-93e1-c09c72f5dcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:32.373053 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dd32c-036d-46a3-93e1-c09c72f5dcd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:32.373467 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dd32c-036d-46a3-93e1-c09c72f5dcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:32.373736 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dd32c-036d-46a3-93e1-c09c72f5dcd8 admin admin] PATH_INFO: `/v3/users/4d11c23f121f474091d7ae232505ece4` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:32.374949 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352dd32c-036d-46a3-93e1-c09c72f5dcd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d11c23f121f474091d7ae232505ece4)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:32.376877 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352dd32c-036d-46a3-93e1-c09c72f5dcd8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:32.619597 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-352dd32c-036d-46a3-93e1-c09c72f5dcd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d11c23f121f474091d7ae232505ece4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:32.632430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-352dd32c-036d-46a3-93e1-c09c72f5dcd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d11c23f121f474091d7ae232505ece4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:32.724957 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-352dd32c-036d-46a3-93e1-c09c72f5dcd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d11c23f121f474091d7ae232505ece4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:32.725615 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-352dd32c-036d-46a3-93e1-c09c72f5dcd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d11c23f121f474091d7ae232505ece4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:32.733790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-352dd32c-036d-46a3-93e1-c09c72f5dcd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d11c23f121f474091d7ae232505ece4'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:32.739779 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1422/2813] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:53:32 2022] DELETE /identity/v3/users/4d11c23f121f474091d7ae232505ece4 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:32.744374 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e99f36-cb61-4052-a141-579c2d01534e None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:32.744856 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e99f36-cb61-4052-a141-579c2d01534e None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:32.744966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e99f36-cb61-4052-a141-579c2d01534e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:32.763854 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-64e99f36-cb61-4052-a141-579c2d01534e None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:32.765652 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-64e99f36-cb61-4052-a141-579c2d01534e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:32.831841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1392/2814] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:32.842425 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dd32c-036d-46a3-93e1-c09c72f5dcd8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:32.846243 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-352dd32c-036d-46a3-93e1-c09c72f5dcd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:32.886113 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:32.904410 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dd32c-036d-46a3-93e1-c09c72f5dcd8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:32.908068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc0828-b65c-4b08-a4bb-0c920548250d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:32.910028 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc0828-b65c-4b08-a4bb-0c920548250d service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:32.910028 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc0828-b65c-4b08-a4bb-0c920548250d service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:32.910028 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc0828-b65c-4b08-a4bb-0c920548250d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:32.918674 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc0828-b65c-4b08-a4bb-0c920548250d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:32.920154 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc0828-b65c-4b08-a4bb-0c920548250d service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:32.929974 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1423/2815] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:53:33.362214 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e99f36-cb61-4052-a141-579c2d01534e None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:33.363678 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-64e99f36-cb61-4052-a141-579c2d01534e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:33.376656 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e99f36-cb61-4052-a141-579c2d01534e None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:33.381784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:33.382680 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:33.382967 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:33.383368 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc admin admin] PATH_INFO: `/v3/projects/60721ae81b9e4141bc7275fe248235fb` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:33.384526 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=60721ae81b9e4141bc7275fe248235fb)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:33.386595 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:33.509001 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60721ae81b9e4141bc7275fe248235fb', 'request_id': 'req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:33.514555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60721ae81b9e4141bc7275fe248235fb', 'request_id': 'req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:33.520684 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60721ae81b9e4141bc7275fe248235fb', 'request_id': 'req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:33.649994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc admin admin] The token cache is being invalidate because project 60721ae81b9e4141bc7275fe248235fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:53:33.650198 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:33.650828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:33.652380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1393/2816] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:53:33 2022] DELETE /identity/v3/projects/60721ae81b9e4141bc7275fe248235fb => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:53:33.657175 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc0828-b65c-4b08-a4bb-0c920548250d service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:33.666940 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc0828-b65c-4b08-a4bb-0c920548250d service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:33.731536 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f760b-ec38-475a-b5d8-f7b108fe7aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:33.733325 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f760b-ec38-475a-b5d8-f7b108fe7aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:33.733646 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f760b-ec38-475a-b5d8-f7b108fe7aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:33.733923 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f760b-ec38-475a-b5d8-f7b108fe7aa1 admin admin] PATH_INFO: `/v3/users/779f44712ddf41f5bbf518c6b53c3feb` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:33.735106 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f760b-ec38-475a-b5d8-f7b108fe7aa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=779f44712ddf41f5bbf518c6b53c3feb)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:33.737003 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f760b-ec38-475a-b5d8-f7b108fe7aa1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:33.924053 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a96f760b-ec38-475a-b5d8-f7b108fe7aa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '779f44712ddf41f5bbf518c6b53c3feb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:33.937051 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a96f760b-ec38-475a-b5d8-f7b108fe7aa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '779f44712ddf41f5bbf518c6b53c3feb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:34.027984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a96f760b-ec38-475a-b5d8-f7b108fe7aa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '779f44712ddf41f5bbf518c6b53c3feb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:34.029269 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a96f760b-ec38-475a-b5d8-f7b108fe7aa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '779f44712ddf41f5bbf518c6b53c3feb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:34.036937 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a96f760b-ec38-475a-b5d8-f7b108fe7aa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '779f44712ddf41f5bbf518c6b53c3feb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:34.043498 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1424/2817] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:53:33 2022] DELETE /identity/v3/users/779f44712ddf41f5bbf518c6b53c3feb => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:34.538060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:34.540661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:34.553696 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0711d82-4bc6-45eb-b438-a0ec5b9c1dbc admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:34.612981 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d244c8-b1a1-4aaa-8933-c7a0dbd00e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:34.613876 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d244c8-b1a1-4aaa-8933-c7a0dbd00e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:34.614164 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d244c8-b1a1-4aaa-8933-c7a0dbd00e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:34.614443 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d244c8-b1a1-4aaa-8933-c7a0dbd00e5a admin admin] PATH_INFO: `/v3/projects/d35a48109c5e4b90a2a53cf0e21ececb` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:34.615774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d244c8-b1a1-4aaa-8933-c7a0dbd00e5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d35a48109c5e4b90a2a53cf0e21ececb)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:34.617838 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d244c8-b1a1-4aaa-8933-c7a0dbd00e5a admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:34.641553 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:34.776866 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-22d244c8-b1a1-4aaa-8933-c7a0dbd00e5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd35a48109c5e4b90a2a53cf0e21ececb', 'request_id': 'req-22d244c8-b1a1-4aaa-8933-c7a0dbd00e5a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:34.782938 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-22d244c8-b1a1-4aaa-8933-c7a0dbd00e5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd35a48109c5e4b90a2a53cf0e21ececb', 'request_id': 'req-22d244c8-b1a1-4aaa-8933-c7a0dbd00e5a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:34.789000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-22d244c8-b1a1-4aaa-8933-c7a0dbd00e5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd35a48109c5e4b90a2a53cf0e21ececb', 'request_id': 'req-22d244c8-b1a1-4aaa-8933-c7a0dbd00e5a'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:34.855906 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-22d244c8-b1a1-4aaa-8933-c7a0dbd00e5a admin admin] The token cache is being invalidate because project d35a48109c5e4b90a2a53cf0e21ececb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:53:34.856195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-22d244c8-b1a1-4aaa-8933-c7a0dbd00e5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:34.857648 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-22d244c8-b1a1-4aaa-8933-c7a0dbd00e5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:34.858961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1394/2818] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:53:34 2022] DELETE /identity/v3/projects/d35a48109c5e4b90a2a53cf0e21ececb => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:34.863829 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43968d-be8a-4b33-b9ba-fc04b0166741 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:34.865611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43968d-be8a-4b33-b9ba-fc04b0166741 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:34.865611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43968d-be8a-4b33-b9ba-fc04b0166741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:34.882639 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ff43968d-be8a-4b33-b9ba-fc04b0166741 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:34.883906 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ff43968d-be8a-4b33-b9ba-fc04b0166741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:34.935535 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:34.946732 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1425/2819] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:34.956291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94a2c7-128e-4312-8b3d-0dacba642ae6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:34.956587 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94a2c7-128e-4312-8b3d-0dacba642ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:34.956876 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94a2c7-128e-4312-8b3d-0dacba642ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:34.975848 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-ed94a2c7-128e-4312-8b3d-0dacba642ae6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:34.977165 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-ed94a2c7-128e-4312-8b3d-0dacba642ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:34.986548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1395/2820] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:34.990945 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43968d-be8a-4b33-b9ba-fc04b0166741 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:35.001813 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43968d-be8a-4b33-b9ba-fc04b0166741 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:35.006439 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35b1de-cc74-4b34-b488-186f0c71db05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:35.007087 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35b1de-cc74-4b34-b488-186f0c71db05 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:35.007373 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35b1de-cc74-4b34-b488-186f0c71db05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:35.007650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35b1de-cc74-4b34-b488-186f0c71db05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:35.008367 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35b1de-cc74-4b34-b488-186f0c71db05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:53:35.008682 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35b1de-cc74-4b34-b488-186f0c71db05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:35.010301 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35b1de-cc74-4b34-b488-186f0c71db05 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:35.024420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1426/2821] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:53:35.026841 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94a2c7-128e-4312-8b3d-0dacba642ae6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:35.037856 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94a2c7-128e-4312-8b3d-0dacba642ae6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:35.042424 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c4cac-038b-4d62-921b-cba44a6f545d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:35.043080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c4cac-038b-4d62-921b-cba44a6f545d admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:35.043371 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c4cac-038b-4d62-921b-cba44a6f545d admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:35.043700 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c4cac-038b-4d62-921b-cba44a6f545d admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:35.045013 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c4cac-038b-4d62-921b-cba44a6f545d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:35.047038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c4cac-038b-4d62-921b-cba44a6f545d admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:35.178030 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1396/2822] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:53:35 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:53:35.182361 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35b1de-cc74-4b34-b488-186f0c71db05 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:35.192403 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35b1de-cc74-4b34-b488-186f0c71db05 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:35.248668 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1844a-6fe8-4b86-b9b5-d1be3ca9b33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:35.249371 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1844a-6fe8-4b86-b9b5-d1be3ca9b33f admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:35.249667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1844a-6fe8-4b86-b9b5-d1be3ca9b33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:35.249954 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1844a-6fe8-4b86-b9b5-d1be3ca9b33f admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:35.250663 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1844a-6fe8-4b86-b9b5-d1be3ca9b33f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:35.252620 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1844a-6fe8-4b86-b9b5-d1be3ca9b33f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:35.358872 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1427/2823] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:53:35 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:53:35.362929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c4cac-038b-4d62-921b-cba44a6f545d admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:35.373318 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c4cac-038b-4d62-921b-cba44a6f545d admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:35.378139 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c50d1-fac7-4ef4-b429-e589c3f3c653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:35.378862 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c50d1-fac7-4ef4-b429-e589c3f3c653 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:35.379151 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c50d1-fac7-4ef4-b429-e589c3f3c653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:35.379431 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c50d1-fac7-4ef4-b429-e589c3f3c653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:35.380161 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c50d1-fac7-4ef4-b429-e589c3f3c653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:35.381812 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c50d1-fac7-4ef4-b429-e589c3f3c653 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:35.392747 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1397/2824] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:35.396664 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1844a-6fe8-4b86-b9b5-d1be3ca9b33f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:35.407272 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1844a-6fe8-4b86-b9b5-d1be3ca9b33f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:35.411851 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41420f81-4c50-41e0-bde8-90fe9b38fdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:35.412500 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41420f81-4c50-41e0-bde8-90fe9b38fdac admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:35.412834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41420f81-4c50-41e0-bde8-90fe9b38fdac admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:35.413084 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41420f81-4c50-41e0-bde8-90fe9b38fdac admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:35.413817 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41420f81-4c50-41e0-bde8-90fe9b38fdac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:35.415391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41420f81-4c50-41e0-bde8-90fe9b38fdac admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:35.426047 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1428/2825] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:35.430150 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c50d1-fac7-4ef4-b429-e589c3f3c653 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:35.440431 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c50d1-fac7-4ef4-b429-e589c3f3c653 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:35.445151 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0826517-47be-43d5-a384-924e5417f21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:35.446021 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0826517-47be-43d5-a384-924e5417f21f admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:35.446309 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0826517-47be-43d5-a384-924e5417f21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:35.446592 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0826517-47be-43d5-a384-924e5417f21f admin admin] PATH_INFO: `/v3/projects/372562c3f4ad4c1e9a3ad36c98261514/users/92493f59a9ed49d8be247d2b0127e717/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:35.462069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0826517-47be-43d5-a384-924e5417f21f admin admin] RBAC: Authorizing `identity:create_grant(project_id=372562c3f4ad4c1e9a3ad36c98261514, user_id=92493f59a9ed49d8be247d2b0127e717, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:35.464774 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0826517-47be-43d5-a384-924e5417f21f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:35.583352 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1398/2826] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:53:35 2022] PUT /identity/v3/projects/372562c3f4ad4c1e9a3ad36c98261514/users/92493f59a9ed49d8be247d2b0127e717/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:35.594917 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662efb8d-de98-4edc-9cfb-de2ac3d3a162 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:35.595211 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662efb8d-de98-4edc-9cfb-de2ac3d3a162 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:35.595490 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662efb8d-de98-4edc-9cfb-de2ac3d3a162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:35.621859 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-662efb8d-de98-4edc-9cfb-de2ac3d3a162 None None] MFA Rules not processed for user `92493f59a9ed49d8be247d2b0127e717`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:35.623025 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-662efb8d-de98-4edc-9cfb-de2ac3d3a162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:35.684980 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1429/2827] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:35.696380 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0826517-47be-43d5-a384-924e5417f21f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:35.701476 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d0826517-47be-43d5-a384-924e5417f21f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:35.759107 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0826517-47be-43d5-a384-924e5417f21f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:35.763433 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444ab8c-88d9-49f9-b8cb-93d8f2f73296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:35.764006 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444ab8c-88d9-49f9-b8cb-93d8f2f73296 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:35.764295 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444ab8c-88d9-49f9-b8cb-93d8f2f73296 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:35.764574 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444ab8c-88d9-49f9-b8cb-93d8f2f73296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:35.773046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444ab8c-88d9-49f9-b8cb-93d8f2f73296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:35.774511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444ab8c-88d9-49f9-b8cb-93d8f2f73296 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:35.783904 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1399/2828] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:53:35.803466 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662efb8d-de98-4edc-9cfb-de2ac3d3a162 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:35.856101 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:35.867166 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662efb8d-de98-4edc-9cfb-de2ac3d3a162 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:35.871841 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9553e08-dd0d-46a7-8e4b-8149fc051d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:35.873437 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9553e08-dd0d-46a7-8e4b-8149fc051d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:35.873676 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9553e08-dd0d-46a7-8e4b-8149fc051d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:35.873959 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9553e08-dd0d-46a7-8e4b-8149fc051d9d admin admin] PATH_INFO: `/v3/users/92493f59a9ed49d8be247d2b0127e717` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:35.875174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9553e08-dd0d-46a7-8e4b-8149fc051d9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=92493f59a9ed49d8be247d2b0127e717)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:35.877125 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9553e08-dd0d-46a7-8e4b-8149fc051d9d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:36.133637 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c9553e08-dd0d-46a7-8e4b-8149fc051d9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92493f59a9ed49d8be247d2b0127e717'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:36.146642 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c9553e08-dd0d-46a7-8e4b-8149fc051d9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92493f59a9ed49d8be247d2b0127e717'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:36.222347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c9553e08-dd0d-46a7-8e4b-8149fc051d9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92493f59a9ed49d8be247d2b0127e717'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:36.222908 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c9553e08-dd0d-46a7-8e4b-8149fc051d9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92493f59a9ed49d8be247d2b0127e717'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:36.230133 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c9553e08-dd0d-46a7-8e4b-8149fc051d9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92493f59a9ed49d8be247d2b0127e717'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:36.236579 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1430/2829] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:53:35 2022] DELETE /identity/v3/users/92493f59a9ed49d8be247d2b0127e717 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:53:36.241017 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b005d78-f5fb-44ee-aedd-1671c071bce0 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:36.241346 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b005d78-f5fb-44ee-aedd-1671c071bce0 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:36.241630 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b005d78-f5fb-44ee-aedd-1671c071bce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:36.260805 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-7b005d78-f5fb-44ee-aedd-1671c071bce0 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:36.262055 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7b005d78-f5fb-44ee-aedd-1671c071bce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:36.304715 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:36.319743 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1400/2830] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:36.329555 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9553e08-dd0d-46a7-8e4b-8149fc051d9d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:36.330973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c9553e08-dd0d-46a7-8e4b-8149fc051d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:36.418773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9553e08-dd0d-46a7-8e4b-8149fc051d9d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:36.423400 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583577c0-9c5b-49e3-852e-cfbd3919a54a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:36.424055 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583577c0-9c5b-49e3-852e-cfbd3919a54a service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:36.424589 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583577c0-9c5b-49e3-852e-cfbd3919a54a service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:36.424879 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583577c0-9c5b-49e3-852e-cfbd3919a54a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:36.434187 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583577c0-9c5b-49e3-852e-cfbd3919a54a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:36.436673 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583577c0-9c5b-49e3-852e-cfbd3919a54a service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:36.446227 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1431/2831] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:53:36.915934 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b005d78-f5fb-44ee-aedd-1671c071bce0 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:36.917646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-7b005d78-f5fb-44ee-aedd-1671c071bce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:36.930966 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b005d78-f5fb-44ee-aedd-1671c071bce0 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:36.939493 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa139ec-0288-4cc2-beef-cbb5023b03ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:36.939852 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa139ec-0288-4cc2-beef-cbb5023b03ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:36.940037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa139ec-0288-4cc2-beef-cbb5023b03ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:36.941613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa139ec-0288-4cc2-beef-cbb5023b03ce admin admin] PATH_INFO: `/v3/projects/372562c3f4ad4c1e9a3ad36c98261514` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:36.942998 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa139ec-0288-4cc2-beef-cbb5023b03ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=372562c3f4ad4c1e9a3ad36c98261514)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:36.945303 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa139ec-0288-4cc2-beef-cbb5023b03ce admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:37.063453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1aa139ec-0288-4cc2-beef-cbb5023b03ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '372562c3f4ad4c1e9a3ad36c98261514', 'request_id': 'req-1aa139ec-0288-4cc2-beef-cbb5023b03ce'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:37.068942 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1aa139ec-0288-4cc2-beef-cbb5023b03ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '372562c3f4ad4c1e9a3ad36c98261514', 'request_id': 'req-1aa139ec-0288-4cc2-beef-cbb5023b03ce'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:37.076529 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1aa139ec-0288-4cc2-beef-cbb5023b03ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '372562c3f4ad4c1e9a3ad36c98261514', 'request_id': 'req-1aa139ec-0288-4cc2-beef-cbb5023b03ce'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:37.164821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1aa139ec-0288-4cc2-beef-cbb5023b03ce admin admin] The token cache is being invalidate because project 372562c3f4ad4c1e9a3ad36c98261514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:53:37.165245 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1aa139ec-0288-4cc2-beef-cbb5023b03ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:37.165873 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-1aa139ec-0288-4cc2-beef-cbb5023b03ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:37.167422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1401/2832] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:53:36 2022] DELETE /identity/v3/projects/372562c3f4ad4c1e9a3ad36c98261514 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:37.172963 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08a169-61a8-4dcc-817f-9f3fde65e3f9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:37.173487 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08a169-61a8-4dcc-817f-9f3fde65e3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:37.173602 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08a169-61a8-4dcc-817f-9f3fde65e3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:37.192132 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0c08a169-61a8-4dcc-817f-9f3fde65e3f9 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:37.193364 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0c08a169-61a8-4dcc-817f-9f3fde65e3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:37.251081 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1432/2833] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:37.258838 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a76ce-e1cc-4338-8e51-e4262f5ebe36 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:37.259126 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a76ce-e1cc-4338-8e51-e4262f5ebe36 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:37.259402 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a76ce-e1cc-4338-8e51-e4262f5ebe36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:37.279149 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-478a76ce-e1cc-4338-8e51-e4262f5ebe36 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:37.280367 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-478a76ce-e1cc-4338-8e51-e4262f5ebe36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:37.288080 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1402/2834] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:37.292205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08a169-61a8-4dcc-817f-9f3fde65e3f9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:37.302108 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08a169-61a8-4dcc-817f-9f3fde65e3f9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:37.306566 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037debf-0454-407c-b6f4-5a076ed24cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:37.307213 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037debf-0454-407c-b6f4-5a076ed24cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:37.307499 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037debf-0454-407c-b6f4-5a076ed24cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:37.307780 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037debf-0454-407c-b6f4-5a076ed24cd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:37.308498 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037debf-0454-407c-b6f4-5a076ed24cd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:53:37.308814 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037debf-0454-407c-b6f4-5a076ed24cd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:37.310595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037debf-0454-407c-b6f4-5a076ed24cd8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:37.322068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1433/2835] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:53:37.326941 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a76ce-e1cc-4338-8e51-e4262f5ebe36 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:37.336167 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a76ce-e1cc-4338-8e51-e4262f5ebe36 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:37.341673 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed854fce-8328-4099-8b7b-662ee0f86d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:37.342333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed854fce-8328-4099-8b7b-662ee0f86d9c admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:37.342626 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed854fce-8328-4099-8b7b-662ee0f86d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:37.342912 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed854fce-8328-4099-8b7b-662ee0f86d9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:37.343646 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed854fce-8328-4099-8b7b-662ee0f86d9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:37.345631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed854fce-8328-4099-8b7b-662ee0f86d9c admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:37.435788 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1403/2836] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:53:37 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:53:37.439948 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037debf-0454-407c-b6f4-5a076ed24cd8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:37.445411 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:37.451815 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037debf-0454-407c-b6f4-5a076ed24cd8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:37.509509 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e9737-98f1-4907-9270-f982e3f6163f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:37.510101 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e9737-98f1-4907-9270-f982e3f6163f admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:37.510387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e9737-98f1-4907-9270-f982e3f6163f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:37.510666 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e9737-98f1-4907-9270-f982e3f6163f admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:37.511359 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e9737-98f1-4907-9270-f982e3f6163f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:37.514035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e9737-98f1-4907-9270-f982e3f6163f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:37.598860 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1434/2837] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:53:37 2022] POST /identity/v3/users => generated 291 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:53:37.603064 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed854fce-8328-4099-8b7b-662ee0f86d9c admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:37.613711 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed854fce-8328-4099-8b7b-662ee0f86d9c admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:37.618594 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d190f-61b7-4076-81ed-207fc9d09f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:37.619249 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d190f-61b7-4076-81ed-207fc9d09f5f admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:37.619538 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d190f-61b7-4076-81ed-207fc9d09f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:37.619826 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d190f-61b7-4076-81ed-207fc9d09f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:37.620558 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d190f-61b7-4076-81ed-207fc9d09f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:37.623375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d190f-61b7-4076-81ed-207fc9d09f5f admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:37.633764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1404/2838] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:37.638498 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e9737-98f1-4907-9270-f982e3f6163f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:37.648332 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e9737-98f1-4907-9270-f982e3f6163f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:37.653169 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb4f4e-1464-4324-9425-9b0c653ad497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:37.654053 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb4f4e-1464-4324-9425-9b0c653ad497 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:37.654053 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb4f4e-1464-4324-9425-9b0c653ad497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:37.654317 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb4f4e-1464-4324-9425-9b0c653ad497 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:37.655038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb4f4e-1464-4324-9425-9b0c653ad497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:37.656619 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb4f4e-1464-4324-9425-9b0c653ad497 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:37.666440 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1435/2839] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:37.670512 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d190f-61b7-4076-81ed-207fc9d09f5f admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:37.680523 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d190f-61b7-4076-81ed-207fc9d09f5f admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:37.685634 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7630d-321e-4706-88a2-08c73db11d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:37.686498 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7630d-321e-4706-88a2-08c73db11d61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:37.686783 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7630d-321e-4706-88a2-08c73db11d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:37.687061 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7630d-321e-4706-88a2-08c73db11d61 admin admin] PATH_INFO: `/v3/projects/4e623039768247c3b344e7e8a9ca143a/users/921092933e044d4b8d224c278b142f9c/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:37.702382 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f7630d-321e-4706-88a2-08c73db11d61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e623039768247c3b344e7e8a9ca143a, user_id=921092933e044d4b8d224c278b142f9c, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:37.705114 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f7630d-321e-4706-88a2-08c73db11d61 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:37.817142 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1405/2840] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:53:37 2022] PUT /identity/v3/projects/4e623039768247c3b344e7e8a9ca143a/users/921092933e044d4b8d224c278b142f9c/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:37.828005 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34ee38-7d90-410c-900f-50012d88df96 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:37.828298 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34ee38-7d90-410c-900f-50012d88df96 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:37.828586 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34ee38-7d90-410c-900f-50012d88df96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:37.854161 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7d34ee38-7d90-410c-900f-50012d88df96 None None] MFA Rules not processed for user `921092933e044d4b8d224c278b142f9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:37.855363 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7d34ee38-7d90-410c-900f-50012d88df96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:37.922240 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1436/2841] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:37.933401 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7630d-321e-4706-88a2-08c73db11d61 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:37.935909 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-52f7630d-321e-4706-88a2-08c73db11d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:37.991421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7630d-321e-4706-88a2-08c73db11d61 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:37.995068 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235adfa0-d054-419d-adc4-0ee9971c943c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:37.995714 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235adfa0-d054-419d-adc4-0ee9971c943c service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:37.996000 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235adfa0-d054-419d-adc4-0ee9971c943c service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:37.996281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235adfa0-d054-419d-adc4-0ee9971c943c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:38.005291 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235adfa0-d054-419d-adc4-0ee9971c943c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:38.006735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235adfa0-d054-419d-adc4-0ee9971c943c service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:38.015447 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1406/2842] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:53:45.006978 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34ee38-7d90-410c-900f-50012d88df96 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:45.008450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7d34ee38-7d90-410c-900f-50012d88df96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:45.066270 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34ee38-7d90-410c-900f-50012d88df96 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:45.069713 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9d27d-055a-4872-a244-cf58b6ece057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:45.070363 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9d27d-055a-4872-a244-cf58b6ece057 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:45.070650 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9d27d-055a-4872-a244-cf58b6ece057 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:45.070931 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9d27d-055a-4872-a244-cf58b6ece057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:45.072665 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-87f9d27d-055a-4872-a244-cf58b6ece057 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:45.099460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:45.137274 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9d27d-055a-4872-a244-cf58b6ece057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:45.142142 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9d27d-055a-4872-a244-cf58b6ece057 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:45.150887 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1437/2843] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:53:48.011453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:48.212302 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235adfa0-d054-419d-adc4-0ee9971c943c service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:48.287815 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235adfa0-d054-419d-adc4-0ee9971c943c service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:48.292643 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934ae71-ec2d-4e76-963e-0f4d68849e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:48.293832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934ae71-ec2d-4e76-963e-0f4d68849e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:48.293832 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934ae71-ec2d-4e76-963e-0f4d68849e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:48.294131 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934ae71-ec2d-4e76-963e-0f4d68849e9e admin admin] PATH_INFO: `/v3/users/921092933e044d4b8d224c278b142f9c` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:48.295653 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934ae71-ec2d-4e76-963e-0f4d68849e9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=921092933e044d4b8d224c278b142f9c)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:48.297671 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934ae71-ec2d-4e76-963e-0f4d68849e9e admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:48.487764 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8934ae71-ec2d-4e76-963e-0f4d68849e9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '921092933e044d4b8d224c278b142f9c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:48.488358 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8934ae71-ec2d-4e76-963e-0f4d68849e9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '921092933e044d4b8d224c278b142f9c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:48.488986 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8934ae71-ec2d-4e76-963e-0f4d68849e9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '921092933e044d4b8d224c278b142f9c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:48.496444 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8934ae71-ec2d-4e76-963e-0f4d68849e9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '921092933e044d4b8d224c278b142f9c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:48.508692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-8934ae71-ec2d-4e76-963e-0f4d68849e9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '921092933e044d4b8d224c278b142f9c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:48.748785 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1407/2844] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:53:48 2022] DELETE /identity/v3/users/921092933e044d4b8d224c278b142f9c => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:48.753518 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdc369-c43c-4e0f-860d-3ab6f0d3ce93 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:48.754029 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdc369-c43c-4e0f-860d-3ab6f0d3ce93 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:48.754135 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdc369-c43c-4e0f-860d-3ab6f0d3ce93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:48.772818 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-69cdc369-c43c-4e0f-860d-3ab6f0d3ce93 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:48.774312 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-69cdc369-c43c-4e0f-860d-3ab6f0d3ce93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:48.836823 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1438/2845] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:53:48.846600 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934ae71-ec2d-4e76-963e-0f4d68849e9e admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:48.848086 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-8934ae71-ec2d-4e76-963e-0f4d68849e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:48.905913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934ae71-ec2d-4e76-963e-0f4d68849e9e admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:48.909515 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b78aab-2d1e-4661-9281-d212d2bfbf95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:48.910171 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b78aab-2d1e-4661-9281-d212d2bfbf95 service neutron] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:48.910461 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b78aab-2d1e-4661-9281-d212d2bfbf95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:48.910741 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b78aab-2d1e-4661-9281-d212d2bfbf95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:48.920975 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b78aab-2d1e-4661-9281-d212d2bfbf95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:48.922482 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b78aab-2d1e-4661-9281-d212d2bfbf95 service neutron] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:48.932200 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1408/2846] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:53:49.367560 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdc369-c43c-4e0f-860d-3ab6f0d3ce93 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:49.369218 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-69cdc369-c43c-4e0f-860d-3ab6f0d3ce93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:49.387962 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdc369-c43c-4e0f-860d-3ab6f0d3ce93 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:49.394973 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17273933-ea30-4f05-b9ea-9e033136fc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:49.401163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17273933-ea30-4f05-b9ea-9e033136fc74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:49.401163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17273933-ea30-4f05-b9ea-9e033136fc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:49.401163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17273933-ea30-4f05-b9ea-9e033136fc74 admin admin] PATH_INFO: `/v3/projects/4e623039768247c3b344e7e8a9ca143a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:49.401163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17273933-ea30-4f05-b9ea-9e033136fc74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e623039768247c3b344e7e8a9ca143a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:49.402969 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17273933-ea30-4f05-b9ea-9e033136fc74 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:49.577392 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-17273933-ea30-4f05-b9ea-9e033136fc74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e623039768247c3b344e7e8a9ca143a', 'request_id': 'req-17273933-ea30-4f05-b9ea-9e033136fc74'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:49.583906 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-17273933-ea30-4f05-b9ea-9e033136fc74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e623039768247c3b344e7e8a9ca143a', 'request_id': 'req-17273933-ea30-4f05-b9ea-9e033136fc74'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:49.741819 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-17273933-ea30-4f05-b9ea-9e033136fc74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e623039768247c3b344e7e8a9ca143a', 'request_id': 'req-17273933-ea30-4f05-b9ea-9e033136fc74'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:49.753709 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-17273933-ea30-4f05-b9ea-9e033136fc74 admin admin] The token cache is being invalidate because project 4e623039768247c3b344e7e8a9ca143a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:53:49.754308 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-17273933-ea30-4f05-b9ea-9e033136fc74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:49.754659 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-17273933-ea30-4f05-b9ea-9e033136fc74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:53:49.756234 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1439/2847] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:53:49 2022] DELETE /identity/v3/projects/4e623039768247c3b344e7e8a9ca143a => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:49.761081 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5588b-37ae-4599-9da9-a7d6d634b5e1 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:49.761556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5588b-37ae-4599-9da9-a7d6d634b5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:49.764297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5588b-37ae-4599-9da9-a7d6d634b5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:49.781973 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-02d5588b-37ae-4599-9da9-a7d6d634b5e1 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:49.783193 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-02d5588b-37ae-4599-9da9-a7d6d634b5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:49.800839 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:49.842452 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1409/2848] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:49.850782 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457184a0-750f-4d6c-933b-3d37ec3b269a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:49.851087 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457184a0-750f-4d6c-933b-3d37ec3b269a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:49.851377 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457184a0-750f-4d6c-933b-3d37ec3b269a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:49.872036 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-457184a0-750f-4d6c-933b-3d37ec3b269a None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:49.873956 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-457184a0-750f-4d6c-933b-3d37ec3b269a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:49.882093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1440/2849] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:49.886615 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5588b-37ae-4599-9da9-a7d6d634b5e1 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:49.898533 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5588b-37ae-4599-9da9-a7d6d634b5e1 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:49.903631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab566b45-4de2-40d3-ac1c-0c711d5b0dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:49.905629 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab566b45-4de2-40d3-ac1c-0c711d5b0dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:49.905918 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab566b45-4de2-40d3-ac1c-0c711d5b0dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:49.906203 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab566b45-4de2-40d3-ac1c-0c711d5b0dfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:49.907253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab566b45-4de2-40d3-ac1c-0c711d5b0dfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:53:49.907253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab566b45-4de2-40d3-ac1c-0c711d5b0dfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:49.908894 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab566b45-4de2-40d3-ac1c-0c711d5b0dfe admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:49.922701 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1410/2850] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:53:49.927075 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457184a0-750f-4d6c-933b-3d37ec3b269a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:49.937014 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457184a0-750f-4d6c-933b-3d37ec3b269a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:49.941745 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ae787-c8b4-481a-9354-85762113c4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:49.945869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ae787-c8b4-481a-9354-85762113c4e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:49.945869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ae787-c8b4-481a-9354-85762113c4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:49.945869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ae787-c8b4-481a-9354-85762113c4e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:49.945869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ae787-c8b4-481a-9354-85762113c4e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:49.945869 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ae787-c8b4-481a-9354-85762113c4e1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:50.040449 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1441/2851] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:53:49 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:53:50.044726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab566b45-4de2-40d3-ac1c-0c711d5b0dfe admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:50.056928 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab566b45-4de2-40d3-ac1c-0c711d5b0dfe admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:50.114038 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4021bde-fd30-46e7-ab82-325f7553bf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:50.114713 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4021bde-fd30-46e7-ab82-325f7553bf14 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:50.115004 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4021bde-fd30-46e7-ab82-325f7553bf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:50.115293 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4021bde-fd30-46e7-ab82-325f7553bf14 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:50.115990 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4021bde-fd30-46e7-ab82-325f7553bf14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:50.117948 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4021bde-fd30-46e7-ab82-325f7553bf14 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:50.222785 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1411/2852] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:53:50 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:53:50.227072 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ae787-c8b4-481a-9354-85762113c4e1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:50.237719 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ae787-c8b4-481a-9354-85762113c4e1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:50.242402 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb46ad7-24c6-4173-9a2e-7862a9b3b748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:50.243063 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb46ad7-24c6-4173-9a2e-7862a9b3b748 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:50.243380 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb46ad7-24c6-4173-9a2e-7862a9b3b748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:50.243662 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb46ad7-24c6-4173-9a2e-7862a9b3b748 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:50.244382 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb46ad7-24c6-4173-9a2e-7862a9b3b748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:50.246010 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb46ad7-24c6-4173-9a2e-7862a9b3b748 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:50.258893 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1442/2853] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:50.267046 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4021bde-fd30-46e7-ab82-325f7553bf14 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:50.277916 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4021bde-fd30-46e7-ab82-325f7553bf14 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:50.283506 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f435d-698c-4f27-853f-95b1ea4d9a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:50.284153 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f435d-698c-4f27-853f-95b1ea4d9a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:50.284446 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f435d-698c-4f27-853f-95b1ea4d9a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:50.284737 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f435d-698c-4f27-853f-95b1ea4d9a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:50.285511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f435d-698c-4f27-853f-95b1ea4d9a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:50.287139 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f435d-698c-4f27-853f-95b1ea4d9a00 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:50.301051 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1412/2854] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:50.302500 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb46ad7-24c6-4173-9a2e-7862a9b3b748 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:50.308984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:50.314060 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb46ad7-24c6-4173-9a2e-7862a9b3b748 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:50.318844 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14c702-3f15-4606-9835-ef7d3f7d089b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:50.319748 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14c702-3f15-4606-9835-ef7d3f7d089b admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:50.320040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14c702-3f15-4606-9835-ef7d3f7d089b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:50.320322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14c702-3f15-4606-9835-ef7d3f7d089b admin admin] PATH_INFO: `/v3/projects/1d666699065342c192157966ec132ec5/users/32b847cb1dfd478694f9719ce26ea08a/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:50.335648 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14c702-3f15-4606-9835-ef7d3f7d089b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d666699065342c192157966ec132ec5, user_id=32b847cb1dfd478694f9719ce26ea08a, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:50.343914 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14c702-3f15-4606-9835-ef7d3f7d089b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:50.441961 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1443/2855] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:53:50 2022] PUT /identity/v3/projects/1d666699065342c192157966ec132ec5/users/32b847cb1dfd478694f9719ce26ea08a/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:50.453422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73340168-f970-4a25-9644-cbf6fd78a5cc None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:50.453725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73340168-f970-4a25-9644-cbf6fd78a5cc None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:50.454015 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73340168-f970-4a25-9644-cbf6fd78a5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:50.481253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-73340168-f970-4a25-9644-cbf6fd78a5cc None None] MFA Rules not processed for user `32b847cb1dfd478694f9719ce26ea08a`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:50.482556 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-73340168-f970-4a25-9644-cbf6fd78a5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:50.548434 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1413/2856] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:50.551746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14c702-3f15-4606-9835-ef7d3f7d089b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:50.614802 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14c702-3f15-4606-9835-ef7d3f7d089b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:50.619464 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a95e00-ec44-4d80-b933-726549aaaeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:50.620131 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a95e00-ec44-4d80-b933-726549aaaeaa admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:50.620419 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a95e00-ec44-4d80-b933-726549aaaeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:50.620708 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a95e00-ec44-4d80-b933-726549aaaeaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:50.621445 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a95e00-ec44-4d80-b933-726549aaaeaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:50.623351 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a95e00-ec44-4d80-b933-726549aaaeaa admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:50.735876 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1444/2857] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:53:50 2022] POST /identity/v3/projects => generated 383 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:53:50.737081 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73340168-f970-4a25-9644-cbf6fd78a5cc None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:50.747991 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73340168-f970-4a25-9644-cbf6fd78a5cc None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:50.761163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:50.806468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cbdd5-b709-447e-9826-24b43790f7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:50.806468 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cbdd5-b709-447e-9826-24b43790f7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:50.806820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cbdd5-b709-447e-9826-24b43790f7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:50.806820 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cbdd5-b709-447e-9826-24b43790f7c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:50.807479 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cbdd5-b709-447e-9826-24b43790f7c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:50.809443 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cbdd5-b709-447e-9826-24b43790f7c2 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:50.914037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1414/2858] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:53:50 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:53:50.918031 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a95e00-ec44-4d80-b933-726549aaaeaa admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:50.929701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a95e00-ec44-4d80-b933-726549aaaeaa admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:50.934405 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df894c0-155a-447f-909d-bdc0f561fec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:50.935068 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df894c0-155a-447f-909d-bdc0f561fec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:50.935354 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df894c0-155a-447f-909d-bdc0f561fec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:50.935701 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df894c0-155a-447f-909d-bdc0f561fec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:50.937038 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df894c0-155a-447f-909d-bdc0f561fec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:50.938794 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df894c0-155a-447f-909d-bdc0f561fec5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:50.950153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1445/2859] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:53:51.007151 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cbdd5-b709-447e-9826-24b43790f7c2 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:51.017943 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cbdd5-b709-447e-9826-24b43790f7c2 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:51.023480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9fbaa1-97f0-44a3-9ba1-9e72a909b215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:51.024276 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fbaa1-97f0-44a3-9ba1-9e72a909b215 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:51.024627 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fbaa1-97f0-44a3-9ba1-9e72a909b215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:51.024915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fbaa1-97f0-44a3-9ba1-9e72a909b215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:51.025692 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9fbaa1-97f0-44a3-9ba1-9e72a909b215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:51.027299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9fbaa1-97f0-44a3-9ba1-9e72a909b215 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:51.042166 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1415/2860] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:51.045387 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df894c0-155a-447f-909d-bdc0f561fec5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:51.056037 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df894c0-155a-447f-909d-bdc0f561fec5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:51.065935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87656ada-8961-4fb3-849e-8dd84d131403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:51.065935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87656ada-8961-4fb3-849e-8dd84d131403 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:51.065935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87656ada-8961-4fb3-849e-8dd84d131403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:51.065935 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87656ada-8961-4fb3-849e-8dd84d131403 admin admin] PATH_INFO: `/v3/projects/3eaec80807634128b8ce2d91d229b58c/users/f39021939fa24e7bb600d89b7999fe27/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:51.078230 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87656ada-8961-4fb3-849e-8dd84d131403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eaec80807634128b8ce2d91d229b58c, user_id=f39021939fa24e7bb600d89b7999fe27, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:51.080977 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87656ada-8961-4fb3-849e-8dd84d131403 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:51.198161 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1446/2861] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:53:51 2022] PUT /identity/v3/projects/3eaec80807634128b8ce2d91d229b58c/users/f39021939fa24e7bb600d89b7999fe27/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:51.203154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a04adc-372b-4e3f-a176-809281908b96 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:51.203728 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a04adc-372b-4e3f-a176-809281908b96 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:51.203870 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a04adc-372b-4e3f-a176-809281908b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:51.239661 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-e6a04adc-372b-4e3f-a176-809281908b96 None None] MFA Rules not processed for user `f39021939fa24e7bb600d89b7999fe27`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:51.244332 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-e6a04adc-372b-4e3f-a176-809281908b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:51.347075 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1416/2862] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:51.351441 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87656ada-8961-4fb3-849e-8dd84d131403 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:51.446897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87656ada-8961-4fb3-849e-8dd84d131403 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:51.452398 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edfa12-47e9-42ce-9c98-a6a9e7c09494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:51.453472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edfa12-47e9-42ce-9c98-a6a9e7c09494 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:51.454030 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edfa12-47e9-42ce-9c98-a6a9e7c09494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:51.454030 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edfa12-47e9-42ce-9c98-a6a9e7c09494 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:51.454811 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edfa12-47e9-42ce-9c98-a6a9e7c09494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:51.456995 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edfa12-47e9-42ce-9c98-a6a9e7c09494 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:51.560115 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1447/2863] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:53:51 2022] POST /identity/v3/projects => generated 383 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:53:51.562412 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a04adc-372b-4e3f-a176-809281908b96 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:51.575297 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a04adc-372b-4e3f-a176-809281908b96 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:51.642310 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455243f-60fc-4f76-8931-5574a0eb30ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:51.642998 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455243f-60fc-4f76-8931-5574a0eb30ce admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:51.643470 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455243f-60fc-4f76-8931-5574a0eb30ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:51.643885 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455243f-60fc-4f76-8931-5574a0eb30ce admin admin] PATH_INFO: `/v3/users` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:51.644822 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455243f-60fc-4f76-8931-5574a0eb30ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:51.647194 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455243f-60fc-4f76-8931-5574a0eb30ce admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:51.762025 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1417/2864] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:53:51 2022] POST /identity/v3/users => generated 298 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:53:51.766514 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edfa12-47e9-42ce-9c98-a6a9e7c09494 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:51.777611 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edfa12-47e9-42ce-9c98-a6a9e7c09494 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:51.784153 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76364ca-e42d-4b32-b85b-f8ed65cf68a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:51.784834 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76364ca-e42d-4b32-b85b-f8ed65cf68a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:51.785489 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76364ca-e42d-4b32-b85b-f8ed65cf68a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:51.785613 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76364ca-e42d-4b32-b85b-f8ed65cf68a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:51.786559 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76364ca-e42d-4b32-b85b-f8ed65cf68a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:51.788350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76364ca-e42d-4b32-b85b-f8ed65cf68a8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:51.802774 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1448/2865] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:53:51.807137 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455243f-60fc-4f76-8931-5574a0eb30ce admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:51.818014 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455243f-60fc-4f76-8931-5574a0eb30ce admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:51.823196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a9a65-2636-4272-9305-d055f69c3cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:51.823821 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a9a65-2636-4272-9305-d055f69c3cea admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:51.824177 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a9a65-2636-4272-9305-d055f69c3cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:51.824525 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a9a65-2636-4272-9305-d055f69c3cea admin admin] PATH_INFO: `/v3/projects/44b12371130240eaa44e7dc53fb74893/users/aed0b23cf90147c89adfa516faa267a8/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:51.839693 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:51.841398 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a9a65-2636-4272-9305-d055f69c3cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b12371130240eaa44e7dc53fb74893, user_id=aed0b23cf90147c89adfa516faa267a8, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:51.845024 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a9a65-2636-4272-9305-d055f69c3cea admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:51.929396 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1418/2866] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:53:51 2022] PUT /identity/v3/projects/44b12371130240eaa44e7dc53fb74893/users/aed0b23cf90147c89adfa516faa267a8/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:53:51.933187 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76364ca-e42d-4b32-b85b-f8ed65cf68a8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:51.940388 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:51.944201 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76364ca-e42d-4b32-b85b-f8ed65cf68a8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:52.001084 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d7a27-261a-454c-ac18-26fcd21e5b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:52.001771 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d7a27-261a-454c-ac18-26fcd21e5b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:52.002065 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d7a27-261a-454c-ac18-26fcd21e5b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:52.002347 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d7a27-261a-454c-ac18-26fcd21e5b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:52.003078 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d7a27-261a-454c-ac18-26fcd21e5b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:52.004678 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d7a27-261a-454c-ac18-26fcd21e5b13 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:52.014390 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1449/2867] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:53:52.019426 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a9a65-2636-4272-9305-d055f69c3cea admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:52.029801 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a9a65-2636-4272-9305-d055f69c3cea admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:52.034572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b044ad-1784-489e-bd05-af66bd896c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:52.035469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b044ad-1784-489e-bd05-af66bd896c28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:52.035757 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b044ad-1784-489e-bd05-af66bd896c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:52.036037 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b044ad-1784-489e-bd05-af66bd896c28 admin admin] PATH_INFO: `/v3/projects/44b12371130240eaa44e7dc53fb74893/users/aed0b23cf90147c89adfa516faa267a8/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:52.038077 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b044ad-1784-489e-bd05-af66bd896c28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b12371130240eaa44e7dc53fb74893, user_id=aed0b23cf90147c89adfa516faa267a8, role_id=cb37a6fc8ec74a0ab2ab82dd5696c78e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:52.040783 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b044ad-1784-489e-bd05-af66bd896c28 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:52.052952 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1419/2868] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:53:52 2022] PUT /identity/v3/projects/44b12371130240eaa44e7dc53fb74893/users/aed0b23cf90147c89adfa516faa267a8/roles/cb37a6fc8ec74a0ab2ab82dd5696c78e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:53:52.064229 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99f1dc-eb48-4d0e-a35e-6bdb2028fcca None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:52.064678 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99f1dc-eb48-4d0e-a35e-6bdb2028fcca None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:52.065054 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99f1dc-eb48-4d0e-a35e-6bdb2028fcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:52.091880 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ce99f1dc-eb48-4d0e-a35e-6bdb2028fcca None None] MFA Rules not processed for user `aed0b23cf90147c89adfa516faa267a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:53:52.093177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ce99f1dc-eb48-4d0e-a35e-6bdb2028fcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:52.160111 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1450/2869] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:53:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:53:52.171825 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b044ad-1784-489e-bd05-af66bd896c28 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:52.173651 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-54b044ad-1784-489e-bd05-af66bd896c28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:52.251885 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b044ad-1784-489e-bd05-af66bd896c28 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:52.255644 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb14645-87e1-4171-8ee8-fdc8368440c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:52.256300 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb14645-87e1-4171-8ee8-fdc8368440c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:52.256591 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb14645-87e1-4171-8ee8-fdc8368440c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:52.256875 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb14645-87e1-4171-8ee8-fdc8368440c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:52.313749 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb14645-87e1-4171-8ee8-fdc8368440c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:52.315210 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb14645-87e1-4171-8ee8-fdc8368440c8 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:52.323994 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1420/2870] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:53:54.319951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99f1dc-eb48-4d0e-a35e-6bdb2028fcca None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:54.321445 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ce99f1dc-eb48-4d0e-a35e-6bdb2028fcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:53:54.333828 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99f1dc-eb48-4d0e-a35e-6bdb2028fcca None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:54.337543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b655f-0c85-4f78-bdd6-1e7c994db43c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:54.338187 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b655f-0c85-4f78-bdd6-1e7c994db43c service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:54.338472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b655f-0c85-4f78-bdd6-1e7c994db43c service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:54.338746 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b655f-0c85-4f78-bdd6-1e7c994db43c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:54.394435 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b655f-0c85-4f78-bdd6-1e7c994db43c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:54.395870 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b655f-0c85-4f78-bdd6-1e7c994db43c service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:54.404810 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1451/2871] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:53:56.059995 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb14645-87e1-4171-8ee8-fdc8368440c8 service cinder] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:53:56.068991 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb14645-87e1-4171-8ee8-fdc8368440c8 service cinder] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:53:56.072617 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90189f5c-ca6e-4089-b507-85846d9a2b07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:53:56.073398 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90189f5c-ca6e-4089-b507-85846d9a2b07 service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:53:56.073687 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90189f5c-ca6e-4089-b507-85846d9a2b07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:53:56.073964 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90189f5c-ca6e-4089-b507-85846d9a2b07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:53:56.083699 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90189f5c-ca6e-4089-b507-85846d9a2b07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:53:56.085163 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90189f5c-ca6e-4089-b507-85846d9a2b07 service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:53:56.095333 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1421/2872] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:53:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:54:01.075854 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b655f-0c85-4f78-bdd6-1e7c994db43c service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:01.103998 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:54:01.139148 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b655f-0c85-4f78-bdd6-1e7c994db43c service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:01.144510 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa65e4-02f4-4721-9759-d6693dca84cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:01.145420 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa65e4-02f4-4721-9759-d6693dca84cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:01.145715 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa65e4-02f4-4721-9759-d6693dca84cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:01.146006 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa65e4-02f4-4721-9759-d6693dca84cd admin admin] PATH_INFO: `/v3/users/32b847cb1dfd478694f9719ce26ea08a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:01.147268 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa65e4-02f4-4721-9759-d6693dca84cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=32b847cb1dfd478694f9719ce26ea08a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:01.149221 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa65e4-02f4-4721-9759-d6693dca84cd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:01.315897 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4afa65e4-02f4-4721-9759-d6693dca84cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32b847cb1dfd478694f9719ce26ea08a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:01.328095 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4afa65e4-02f4-4721-9759-d6693dca84cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32b847cb1dfd478694f9719ce26ea08a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:01.398553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4afa65e4-02f4-4721-9759-d6693dca84cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32b847cb1dfd478694f9719ce26ea08a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:01.399364 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4afa65e4-02f4-4721-9759-d6693dca84cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32b847cb1dfd478694f9719ce26ea08a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:01.407136 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4afa65e4-02f4-4721-9759-d6693dca84cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32b847cb1dfd478694f9719ce26ea08a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:01.413478 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1452/2873] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:54:01 2022] DELETE /identity/v3/users/32b847cb1dfd478694f9719ce26ea08a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:54:01.418099 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8999b2-4636-46c8-8b82-937ac7a3fdc2 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:01.418618 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8999b2-4636-46c8-8b82-937ac7a3fdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:01.418720 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8999b2-4636-46c8-8b82-937ac7a3fdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:01.436638 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-0c8999b2-4636-46c8-8b82-937ac7a3fdc2 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:01.437971 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0c8999b2-4636-46c8-8b82-937ac7a3fdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:01.489680 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:54:01.498108 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1422/2874] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:01.508574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa65e4-02f4-4721-9759-d6693dca84cd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:01.510033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4afa65e4-02f4-4721-9759-d6693dca84cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:01.567337 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa65e4-02f4-4721-9759-d6693dca84cd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:01.570877 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba39e8b-ff8a-4299-bef8-97e319e73cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:01.571527 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba39e8b-ff8a-4299-bef8-97e319e73cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:01.571815 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba39e8b-ff8a-4299-bef8-97e319e73cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:01.572094 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba39e8b-ff8a-4299-bef8-97e319e73cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:01.581446 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba39e8b-ff8a-4299-bef8-97e319e73cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:01.582888 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba39e8b-ff8a-4299-bef8-97e319e73cd1 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:01.592222 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1453/2875] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:54:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:54:01.988482 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8999b2-4636-46c8-8b82-937ac7a3fdc2 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:01.993273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-0c8999b2-4636-46c8-8b82-937ac7a3fdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:02.002060 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8999b2-4636-46c8-8b82-937ac7a3fdc2 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:02.006989 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:02.007847 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:02.008135 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:02.008413 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72 admin admin] PATH_INFO: `/v3/projects/1d666699065342c192157966ec132ec5` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:02.009674 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d666699065342c192157966ec132ec5)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:02.011612 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:02.103390 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d666699065342c192157966ec132ec5', 'request_id': 'req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:02.108608 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d666699065342c192157966ec132ec5', 'request_id': 'req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:02.115376 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d666699065342c192157966ec132ec5', 'request_id': 'req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:02.169440 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72 admin admin] The token cache is being invalidate because project 1d666699065342c192157966ec132ec5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:54:02.169733 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:02.170353 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:02.171911 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1423/2876] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:54:01 2022] DELETE /identity/v3/projects/1d666699065342c192157966ec132ec5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:54:02.177472 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba39e8b-ff8a-4299-bef8-97e319e73cd1 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:02.187335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba39e8b-ff8a-4299-bef8-97e319e73cd1 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:02.231338 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:54:02.243955 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b76eec-e079-41e2-b86e-dcde24579ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:02.244833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b76eec-e079-41e2-b86e-dcde24579ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:02.245164 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b76eec-e079-41e2-b86e-dcde24579ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:02.245450 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b76eec-e079-41e2-b86e-dcde24579ebe admin admin] PATH_INFO: `/v3/users/f39021939fa24e7bb600d89b7999fe27` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:02.246657 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b76eec-e079-41e2-b86e-dcde24579ebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f39021939fa24e7bb600d89b7999fe27)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:02.248577 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b76eec-e079-41e2-b86e-dcde24579ebe admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:02.366437 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-14b76eec-e079-41e2-b86e-dcde24579ebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f39021939fa24e7bb600d89b7999fe27'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:02.378002 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-14b76eec-e079-41e2-b86e-dcde24579ebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f39021939fa24e7bb600d89b7999fe27'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:02.450391 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-14b76eec-e079-41e2-b86e-dcde24579ebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f39021939fa24e7bb600d89b7999fe27'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:02.451008 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-14b76eec-e079-41e2-b86e-dcde24579ebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f39021939fa24e7bb600d89b7999fe27'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:02.458891 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-14b76eec-e079-41e2-b86e-dcde24579ebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f39021939fa24e7bb600d89b7999fe27'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:02.464941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1454/2877] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:54:02 2022] DELETE /identity/v3/users/f39021939fa24e7bb600d89b7999fe27 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:54:02.937160 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:02.940336 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:02.954883 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf2e2f-f02b-40f4-908e-3f4c07b87a72 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:03.021840 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180173d-7a78-4ceb-8725-87d1d30097cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:03.022708 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180173d-7a78-4ceb-8725-87d1d30097cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:03.022998 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180173d-7a78-4ceb-8725-87d1d30097cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:03.023286 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180173d-7a78-4ceb-8725-87d1d30097cf admin admin] PATH_INFO: `/v3/projects/3eaec80807634128b8ce2d91d229b58c` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:03.024474 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180173d-7a78-4ceb-8725-87d1d30097cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eaec80807634128b8ce2d91d229b58c)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:03.026548 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180173d-7a78-4ceb-8725-87d1d30097cf admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:03.130813 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3180173d-7a78-4ceb-8725-87d1d30097cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eaec80807634128b8ce2d91d229b58c', 'request_id': 'req-3180173d-7a78-4ceb-8725-87d1d30097cf'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:03.136087 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3180173d-7a78-4ceb-8725-87d1d30097cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eaec80807634128b8ce2d91d229b58c', 'request_id': 'req-3180173d-7a78-4ceb-8725-87d1d30097cf'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:03.142123 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3180173d-7a78-4ceb-8725-87d1d30097cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eaec80807634128b8ce2d91d229b58c', 'request_id': 'req-3180173d-7a78-4ceb-8725-87d1d30097cf'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:03.227076 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3180173d-7a78-4ceb-8725-87d1d30097cf admin admin] The token cache is being invalidate because project 3eaec80807634128b8ce2d91d229b58c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:54:03.227436 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3180173d-7a78-4ceb-8725-87d1d30097cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:03.228091 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-3180173d-7a78-4ceb-8725-87d1d30097cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:03.229729 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1424/2878] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:54:02 2022] DELETE /identity/v3/projects/3eaec80807634128b8ce2d91d229b58c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:54:03.233698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b76eec-e079-41e2-b86e-dcde24579ebe admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:03.235100 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-14b76eec-e079-41e2-b86e-dcde24579ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:03.248686 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b76eec-e079-41e2-b86e-dcde24579ebe admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:03.308255 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8bc71-360e-4c2d-b5dd-8829cfb80c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:03.309167 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8bc71-360e-4c2d-b5dd-8829cfb80c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:03.309460 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8bc71-360e-4c2d-b5dd-8829cfb80c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:03.309743 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8bc71-360e-4c2d-b5dd-8829cfb80c3b admin admin] PATH_INFO: `/v3/users/aed0b23cf90147c89adfa516faa267a8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:03.310917 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8bc71-360e-4c2d-b5dd-8829cfb80c3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=aed0b23cf90147c89adfa516faa267a8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:03.312842 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8bc71-360e-4c2d-b5dd-8829cfb80c3b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:03.538426 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dfc8bc71-360e-4c2d-b5dd-8829cfb80c3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aed0b23cf90147c89adfa516faa267a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:03.559101 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dfc8bc71-360e-4c2d-b5dd-8829cfb80c3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aed0b23cf90147c89adfa516faa267a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:03.559101 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:54:03.683941 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dfc8bc71-360e-4c2d-b5dd-8829cfb80c3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aed0b23cf90147c89adfa516faa267a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:03.684568 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dfc8bc71-360e-4c2d-b5dd-8829cfb80c3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aed0b23cf90147c89adfa516faa267a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:03.691726 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-dfc8bc71-360e-4c2d-b5dd-8829cfb80c3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aed0b23cf90147c89adfa516faa267a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:03.697967 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1455/2879] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:54:03 2022] DELETE /identity/v3/users/aed0b23cf90147c89adfa516faa267a8 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:54:04.131960 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180173d-7a78-4ceb-8725-87d1d30097cf admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:04.133598 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-3180173d-7a78-4ceb-8725-87d1d30097cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:04.147455 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180173d-7a78-4ceb-8725-87d1d30097cf admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:04.157572 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:54:04.206939 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de233fd3-b4b5-4bde-862b-3bc1b3da7986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:04.207811 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de233fd3-b4b5-4bde-862b-3bc1b3da7986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:04.208090 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de233fd3-b4b5-4bde-862b-3bc1b3da7986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:04.208362 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de233fd3-b4b5-4bde-862b-3bc1b3da7986 admin admin] PATH_INFO: `/v3/projects/44b12371130240eaa44e7dc53fb74893` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:04.209688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de233fd3-b4b5-4bde-862b-3bc1b3da7986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44b12371130240eaa44e7dc53fb74893)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:04.211600 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de233fd3-b4b5-4bde-862b-3bc1b3da7986 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:04.352940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-de233fd3-b4b5-4bde-862b-3bc1b3da7986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b12371130240eaa44e7dc53fb74893', 'request_id': 'req-de233fd3-b4b5-4bde-862b-3bc1b3da7986'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:04.358555 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-de233fd3-b4b5-4bde-862b-3bc1b3da7986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b12371130240eaa44e7dc53fb74893', 'request_id': 'req-de233fd3-b4b5-4bde-862b-3bc1b3da7986'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:04.368726 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-de233fd3-b4b5-4bde-862b-3bc1b3da7986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b12371130240eaa44e7dc53fb74893', 'request_id': 'req-de233fd3-b4b5-4bde-862b-3bc1b3da7986'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:04.507460 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-de233fd3-b4b5-4bde-862b-3bc1b3da7986 admin admin] The token cache is being invalidate because project 44b12371130240eaa44e7dc53fb74893 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:54:04.507752 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-de233fd3-b4b5-4bde-862b-3bc1b3da7986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:04.508377 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-de233fd3-b4b5-4bde-862b-3bc1b3da7986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:04.509946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1425/2880] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:54:04 2022] DELETE /identity/v3/projects/44b12371130240eaa44e7dc53fb74893 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:54:04.514825 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e2da0-9988-4633-ba04-7c7bab09a36f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:04.515335 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e2da0-9988-4633-ba04-7c7bab09a36f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:04.515443 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e2da0-9988-4633-ba04-7c7bab09a36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:04.535177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-530e2da0-9988-4633-ba04-7c7bab09a36f None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:04.549729 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-530e2da0-9988-4633-ba04-7c7bab09a36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:04.594705 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1456/2881] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:04.603874 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63ee67-4b45-4f1e-b41b-50f1bfae45f6 None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:04.604148 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63ee67-4b45-4f1e-b41b-50f1bfae45f6 None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:04.604437 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63ee67-4b45-4f1e-b41b-50f1bfae45f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:04.623311 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-4e63ee67-4b45-4f1e-b41b-50f1bfae45f6 None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:04.624453 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-4e63ee67-4b45-4f1e-b41b-50f1bfae45f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:04.633422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1426/2882] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:04.637716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e2da0-9988-4633-ba04-7c7bab09a36f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:04.647339 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e2da0-9988-4633-ba04-7c7bab09a36f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:04.651833 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3dc80-373a-4e7a-9c78-64b854cb6dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:04.652479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3dc80-373a-4e7a-9c78-64b854cb6dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:04.652764 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3dc80-373a-4e7a-9c78-64b854cb6dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:04.653040 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3dc80-373a-4e7a-9c78-64b854cb6dc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:04.653785 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3dc80-373a-4e7a-9c78-64b854cb6dc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:54:04.654093 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3dc80-373a-4e7a-9c78-64b854cb6dc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:04.655683 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3dc80-373a-4e7a-9c78-64b854cb6dc7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:04.667811 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1457/2883] 10.222.0.64 () {58 vars in 1152 bytes} [Mon May 23 15:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:54:04.671961 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63ee67-4b45-4f1e-b41b-50f1bfae45f6 None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:04.682246 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63ee67-4b45-4f1e-b41b-50f1bfae45f6 None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:04.686940 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9503ef-5fb1-4178-9c0e-467d9b0cf1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:04.687624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9503ef-5fb1-4178-9c0e-467d9b0cf1ef admin admin] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:04.687910 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9503ef-5fb1-4178-9c0e-467d9b0cf1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:04.688191 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9503ef-5fb1-4178-9c0e-467d9b0cf1ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:04.688923 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9503ef-5fb1-4178-9c0e-467d9b0cf1ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:04.691530 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9503ef-5fb1-4178-9c0e-467d9b0cf1ef admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:04.800892 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1427/2884] 10.222.0.64 () {60 vars in 1152 bytes} [Mon May 23 15:54:04 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:54:04.805161 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3dc80-373a-4e7a-9c78-64b854cb6dc7 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:04.816241 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3dc80-373a-4e7a-9c78-64b854cb6dc7 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:04.871200 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0100085-0682-4148-a47a-b5c6f6bb217f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:04.871874 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0100085-0682-4148-a47a-b5c6f6bb217f admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:04.872163 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0100085-0682-4148-a47a-b5c6f6bb217f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:04.872445 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0100085-0682-4148-a47a-b5c6f6bb217f admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:04.873216 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0100085-0682-4148-a47a-b5c6f6bb217f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:04.875174 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0100085-0682-4148-a47a-b5c6f6bb217f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:04.998085 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1458/2885] 10.222.0.64 () {60 vars in 1143 bytes} [Mon May 23 15:54:04 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:54:05.002149 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9503ef-5fb1-4178-9c0e-467d9b0cf1ef admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:05.012387 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9503ef-5fb1-4178-9c0e-467d9b0cf1ef admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:05.017278 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e687a-bbbe-4815-a0ec-b8d5936fc024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:05.018097 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e687a-bbbe-4815-a0ec-b8d5936fc024 admin admin] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:05.018408 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e687a-bbbe-4815-a0ec-b8d5936fc024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:05.018688 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e687a-bbbe-4815-a0ec-b8d5936fc024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:05.019425 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e687a-bbbe-4815-a0ec-b8d5936fc024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:05.021049 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e687a-bbbe-4815-a0ec-b8d5936fc024 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:05.031279 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1428/2886] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:54:05.035903 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0100085-0682-4148-a47a-b5c6f6bb217f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:05.045806 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0100085-0682-4148-a47a-b5c6f6bb217f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:05.051328 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1d04d-c3dc-4f55-9b49-f64d64a6fc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:05.051975 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1d04d-c3dc-4f55-9b49-f64d64a6fc0b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:05.052261 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1d04d-c3dc-4f55-9b49-f64d64a6fc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:05.052543 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1d04d-c3dc-4f55-9b49-f64d64a6fc0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:05.053300 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1d04d-c3dc-4f55-9b49-f64d64a6fc0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:05.054951 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1d04d-c3dc-4f55-9b49-f64d64a6fc0b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:05.064790 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1459/2887] 10.222.0.64 () {58 vars in 1121 bytes} [Mon May 23 15:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:54:05.068784 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e687a-bbbe-4815-a0ec-b8d5936fc024 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:05.078631 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e687a-bbbe-4815-a0ec-b8d5936fc024 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:05.083744 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c4d09-5644-4eed-abf0-37f3c494d072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:05.084645 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c4d09-5644-4eed-abf0-37f3c494d072 admin admin] REQUEST_METHOD: `PUT` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:05.084929 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c4d09-5644-4eed-abf0-37f3c494d072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:05.085251 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c4d09-5644-4eed-abf0-37f3c494d072 admin admin] PATH_INFO: `/v3/projects/c2a761e86d4a4ef886863224c6cdf05e/users/1568417535e447f29dbc801bdf0121fa/roles/9330473f62ec4a82a82d1c4beb6160d7` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:05.100421 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c4d09-5644-4eed-abf0-37f3c494d072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2a761e86d4a4ef886863224c6cdf05e, user_id=1568417535e447f29dbc801bdf0121fa, role_id=9330473f62ec4a82a82d1c4beb6160d7)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:05.103111 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c4d09-5644-4eed-abf0-37f3c494d072 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:05.177146 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1429/2888] 10.222.0.64 () {60 vars in 1482 bytes} [Mon May 23 15:54:05 2022] PUT /identity/v3/projects/c2a761e86d4a4ef886863224c6cdf05e/users/1568417535e447f29dbc801bdf0121fa/roles/9330473f62ec4a82a82d1c4beb6160d7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:54:05.187296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0007363-55ad-4c5c-9566-f1f5cdbe4ce6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:05.187595 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0007363-55ad-4c5c-9566-f1f5cdbe4ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:05.187877 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0007363-55ad-4c5c-9566-f1f5cdbe4ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:05.214386 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d0007363-55ad-4c5c-9566-f1f5cdbe4ce6 None None] MFA Rules not processed for user `1568417535e447f29dbc801bdf0121fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:05.215591 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d0007363-55ad-4c5c-9566-f1f5cdbe4ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:05.255892 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:54:05.280033 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1460/2889] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:54:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:05.292273 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c4d09-5644-4eed-abf0-37f3c494d072 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:05.294025 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-030c4d09-5644-4eed-abf0-37f3c494d072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:05.337915 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:54:05.352139 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c4d09-5644-4eed-abf0-37f3c494d072 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:05.355838 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e561ce-39f6-4f32-b93e-1be9e38fc6cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8750b0450bf343f590c52b7d5c68d5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:05.356483 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e561ce-39f6-4f32-b93e-1be9e38fc6cd service cinder] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:05.356768 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e561ce-39f6-4f32-b93e-1be9e38fc6cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:05.357045 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e561ce-39f6-4f32-b93e-1be9e38fc6cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:05.365269 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e561ce-39f6-4f32-b93e-1be9e38fc6cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:05.366729 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e561ce-39f6-4f32-b93e-1be9e38fc6cd service cinder] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:05.375624 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1430/2890] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:54:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:54:15.370042 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=9 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:54:15.456819 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=8 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:54:16.975982 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0007363-55ad-4c5c-9566-f1f5cdbe4ce6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:17.038791 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0007363-55ad-4c5c-9566-f1f5cdbe4ce6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:17.043269 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71218d-25d4-49d8-8a71-785051554f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:17.044145 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71218d-25d4-49d8-8a71-785051554f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:17.044438 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71218d-25d4-49d8-8a71-785051554f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:17.044722 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71218d-25d4-49d8-8a71-785051554f15 admin admin] PATH_INFO: `/v3/users/1568417535e447f29dbc801bdf0121fa` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:17.045981 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa71218d-25d4-49d8-8a71-785051554f15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1568417535e447f29dbc801bdf0121fa)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:17.047907 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa71218d-25d4-49d8-8a71-785051554f15 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:17.246089 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fa71218d-25d4-49d8-8a71-785051554f15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1568417535e447f29dbc801bdf0121fa'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:17.259506 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fa71218d-25d4-49d8-8a71-785051554f15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1568417535e447f29dbc801bdf0121fa'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:17.329853 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fa71218d-25d4-49d8-8a71-785051554f15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1568417535e447f29dbc801bdf0121fa'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:17.330571 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fa71218d-25d4-49d8-8a71-785051554f15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1568417535e447f29dbc801bdf0121fa'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:17.338017 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-fa71218d-25d4-49d8-8a71-785051554f15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1568417535e447f29dbc801bdf0121fa'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:17.344350 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1461/2891] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:54:16 2022] DELETE /identity/v3/users/1568417535e447f29dbc801bdf0121fa => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:54:17.349503 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a91f2f-9799-4067-b8b9-688628eb0b5b None None] REQUEST_METHOD: `POST` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:17.349999 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a91f2f-9799-4067-b8b9-688628eb0b5b None None] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:17.350103 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a91f2f-9799-4067-b8b9-688628eb0b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:17.367946 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.auth.core [None req-b9a91f2f-9799-4067-b8b9-688628eb0b5b None None] MFA Rules not processed for user `2781ce56481d4d179e62cc19ea41a8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:17.369173 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b9a91f2f-9799-4067-b8b9-688628eb0b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:17.426033 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1431/2892] 10.222.0.64 () {58 vars in 957 bytes} [Mon May 23 15:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:17.435547 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71218d-25d4-49d8-8a71-785051554f15 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:17.436940 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fa71218d-25d4-49d8-8a71-785051554f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:17.493479 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71218d-25d4-49d8-8a71-785051554f15 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:17.497127 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229ca5e-80d0-4be7-b7cb-95eb1b75f14c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12ceb8a355a44b9fb29918483379cb38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:17.497716 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229ca5e-80d0-4be7-b7cb-95eb1b75f14c service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:17.498000 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229ca5e-80d0-4be7-b7cb-95eb1b75f14c service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:17.498303 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229ca5e-80d0-4be7-b7cb-95eb1b75f14c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:17.507966 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229ca5e-80d0-4be7-b7cb-95eb1b75f14c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:17.509430 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229ca5e-80d0-4be7-b7cb-95eb1b75f14c service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:17.518735 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1462/2893] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:54:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:54:17.953962 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a91f2f-9799-4067-b8b9-688628eb0b5b None None] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:17.955491 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-b9a91f2f-9799-4067-b8b9-688628eb0b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:17.967511 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a91f2f-9799-4067-b8b9-688628eb0b5b None None] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:17.972780 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:17.973684 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:17.973974 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:17.974299 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace admin admin] PATH_INFO: `/v3/projects/c2a761e86d4a4ef886863224c6cdf05e` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:17.975492 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2a761e86d4a4ef886863224c6cdf05e)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:17.977464 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:18.090036 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2a761e86d4a4ef886863224c6cdf05e', 'request_id': 'req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:18.095594 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2a761e86d4a4ef886863224c6cdf05e', 'request_id': 'req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:18.102575 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2a761e86d4a4ef886863224c6cdf05e', 'request_id': 'req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:18.186281 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace admin admin] The token cache is being invalidate because project c2a761e86d4a4ef886863224c6cdf05e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:54:18.186574 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:18.187196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:54:18.188725 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1432/2894] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:54:17 2022] DELETE /identity/v3/projects/c2a761e86d4a4ef886863224c6cdf05e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:54:27.514588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:54:28.183347 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:54:57.332421 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c564c-0795-477b-9c7b-e58acb77d85e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:57.332928 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c564c-0795-477b-9c7b-e58acb77d85e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:57.333035 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c564c-0795-477b-9c7b-e58acb77d85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:57.389500 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a22c564c-0795-477b-9c7b-e58acb77d85e None None] MFA Rules not processed for user `08dad0888c8542f4aed748a1a35db2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:57.390691 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a22c564c-0795-477b-9c7b-e58acb77d85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:57.461553 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1463/2895] 10.222.0.64 () {58 vars in 967 bytes} [Mon May 23 15:54:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:57.503154 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:57.504613 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:57.596196 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f75f07-20ee-4b63-94fb-a4e7fa0e6ace admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:57.601254 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d742665-3f41-41c7-accf-27c790637af8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8c79c36c7204368816593f0e473191b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:57.601907 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d742665-3f41-41c7-accf-27c790637af8 service swift] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:57.602195 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d742665-3f41-41c7-accf-27c790637af8 service swift] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:57.602480 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d742665-3f41-41c7-accf-27c790637af8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:57.610913 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d742665-3f41-41c7-accf-27c790637af8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:57.612509 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d742665-3f41-41c7-accf-27c790637af8 service swift] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:57.621401 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1433/2896] 10.222.0.64 () {60 vars in 1384 bytes} [Mon May 23 15:54:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:55:05.574667 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c564c-0795-477b-9c7b-e58acb77d85e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:55:05.576045 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a22c564c-0795-477b-9c7b-e58acb77d85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:55:05.671818 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c564c-0795-477b-9c7b-e58acb77d85e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:55:05.677141 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd0ef3d-faae-407e-a595-92c3713499c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fbaeb7583a456d81cf79f6b101fb1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:55:05.677792 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0ef3d-faae-407e-a595-92c3713499c0 service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:55:05.678080 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0ef3d-faae-407e-a595-92c3713499c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:55:05.678362 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0ef3d-faae-407e-a595-92c3713499c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:55:05.680138 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2cd0ef3d-faae-407e-a595-92c3713499c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:55:05.697874 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd0ef3d-faae-407e-a595-92c3713499c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:55:05.698253 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d742665-3f41-41c7-accf-27c790637af8 service swift] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:55:05.698698 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd0ef3d-faae-407e-a595-92c3713499c0 service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:55:05.698814 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-1d742665-3f41-41c7-accf-27c790637af8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:55:05.710205 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1464/2897] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:55:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:55:05.773799 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d742665-3f41-41c7-accf-27c790637af8 service swift] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:55:05.778629 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b485f0e-6cd9-4281-be97-d249419ff0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:55:05.779507 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b485f0e-6cd9-4281-be97-d249419ff0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:55:05.779791 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b485f0e-6cd9-4281-be97-d249419ff0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:55:05.780069 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b485f0e-6cd9-4281-be97-d249419ff0a4 admin admin] PATH_INFO: `/v3/users/172c28d985cb40409d563fad4fa2ad5c` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:55:05.781272 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b485f0e-6cd9-4281-be97-d249419ff0a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=172c28d985cb40409d563fad4fa2ad5c)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:55:05.783188 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b485f0e-6cd9-4281-be97-d249419ff0a4 admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:55:05.960375 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9b485f0e-6cd9-4281-be97-d249419ff0a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '172c28d985cb40409d563fad4fa2ad5c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:05.961033 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9b485f0e-6cd9-4281-be97-d249419ff0a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '172c28d985cb40409d563fad4fa2ad5c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:05.961669 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9b485f0e-6cd9-4281-be97-d249419ff0a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '172c28d985cb40409d563fad4fa2ad5c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:05.969487 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9b485f0e-6cd9-4281-be97-d249419ff0a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '172c28d985cb40409d563fad4fa2ad5c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:05.981204 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:55:05.982434 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-9b485f0e-6cd9-4281-be97-d249419ff0a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '172c28d985cb40409d563fad4fa2ad5c'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:06.066537 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1434/2898] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:55:05 2022] DELETE /identity/v3/users/172c28d985cb40409d563fad4fa2ad5c => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:55:06.317210 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd0ef3d-faae-407e-a595-92c3713499c0 service placement] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:55:06.328738 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd0ef3d-faae-407e-a595-92c3713499c0 service placement] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:55:06.353786 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:55:06.386542 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f9370-84b9-49b1-a679-c3204459ccbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:55:06.387406 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f9370-84b9-49b1-a679-c3204459ccbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:55:06.387713 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f9370-84b9-49b1-a679-c3204459ccbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:55:06.387988 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f9370-84b9-49b1-a679-c3204459ccbb admin admin] PATH_INFO: `/v3/projects/3daaf17c3dac442f82831c7df811f511` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:55:06.389215 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f9370-84b9-49b1-a679-c3204459ccbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3daaf17c3dac442f82831c7df811f511)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:55:06.391125 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f9370-84b9-49b1-a679-c3204459ccbb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:55:06.504652 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9a4f9370-84b9-49b1-a679-c3204459ccbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3daaf17c3dac442f82831c7df811f511', 'request_id': 'req-9a4f9370-84b9-49b1-a679-c3204459ccbb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:06.512072 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9a4f9370-84b9-49b1-a679-c3204459ccbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3daaf17c3dac442f82831c7df811f511', 'request_id': 'req-9a4f9370-84b9-49b1-a679-c3204459ccbb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:06.601027 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9a4f9370-84b9-49b1-a679-c3204459ccbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3daaf17c3dac442f82831c7df811f511', 'request_id': 'req-9a4f9370-84b9-49b1-a679-c3204459ccbb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:06.612322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9a4f9370-84b9-49b1-a679-c3204459ccbb admin admin] The token cache is being invalidate because project 3daaf17c3dac442f82831c7df811f511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:55:06.616485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9a4f9370-84b9-49b1-a679-c3204459ccbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:06.616485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9a4f9370-84b9-49b1-a679-c3204459ccbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:06.616485 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1465/2899] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:55:06 2022] DELETE /identity/v3/projects/3daaf17c3dac442f82831c7df811f511 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:55:06.618853 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b485f0e-6cd9-4281-be97-d249419ff0a4 admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:55:06.620828 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-9b485f0e-6cd9-4281-be97-d249419ff0a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:55:06.633636 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b485f0e-6cd9-4281-be97-d249419ff0a4 admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:55:06.688833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562c8ba-33e5-4c69-8e33-35a4a0c8e4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:55:06.689732 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562c8ba-33e5-4c69-8e33-35a4a0c8e4de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:55:06.690023 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562c8ba-33e5-4c69-8e33-35a4a0c8e4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:55:06.690301 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562c8ba-33e5-4c69-8e33-35a4a0c8e4de admin admin] PATH_INFO: `/v3/users/7080c136d70e40a7850e54fc8dabde05` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:55:06.691469 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562c8ba-33e5-4c69-8e33-35a4a0c8e4de admin admin] RBAC: Authorizing `identity:delete_user(user_id=7080c136d70e40a7850e54fc8dabde05)` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:55:06.693457 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562c8ba-33e5-4c69-8e33-35a4a0c8e4de admin admin] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:55:06.884168 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d562c8ba-33e5-4c69-8e33-35a4a0c8e4de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7080c136d70e40a7850e54fc8dabde05'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:06.884808 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d562c8ba-33e5-4c69-8e33-35a4a0c8e4de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7080c136d70e40a7850e54fc8dabde05'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:06.886068 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d562c8ba-33e5-4c69-8e33-35a4a0c8e4de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7080c136d70e40a7850e54fc8dabde05'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:06.892735 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d562c8ba-33e5-4c69-8e33-35a4a0c8e4de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7080c136d70e40a7850e54fc8dabde05'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:06.904889 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.notifications [None req-d562c8ba-33e5-4c69-8e33-35a4a0c8e4de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7080c136d70e40a7850e54fc8dabde05'} {{(pid=80598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:06.985992 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1435/2900] 10.222.0.64 () {58 vars in 1223 bytes} [Mon May 23 15:55:06 2022] DELETE /identity/v3/users/7080c136d70e40a7850e54fc8dabde05 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:55:07.220114 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f9370-84b9-49b1-a679-c3204459ccbb admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:55:07.221636 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9a4f9370-84b9-49b1-a679-c3204459ccbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:55:07.238634 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f9370-84b9-49b1-a679-c3204459ccbb admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:55:07.295343 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821794f-9722-45e9-9d56-0db7f50fcff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2781ce56481d4d179e62cc19ea41a8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c9cfa8acbce47f6b0259cbfbf5b338d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:55:07.296322 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821794f-9722-45e9-9d56-0db7f50fcff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:55:07.296661 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821794f-9722-45e9-9d56-0db7f50fcff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:55:07.296984 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821794f-9722-45e9-9d56-0db7f50fcff2 admin admin] PATH_INFO: `/v3/projects/8c7e2b79f5064252a47372718fe656f4` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:55:07.298296 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821794f-9722-45e9-9d56-0db7f50fcff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c7e2b79f5064252a47372718fe656f4)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:55:07.300574 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821794f-9722-45e9-9d56-0db7f50fcff2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:55:07.814424 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3821794f-9722-45e9-9d56-0db7f50fcff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c7e2b79f5064252a47372718fe656f4', 'request_id': 'req-3821794f-9722-45e9-9d56-0db7f50fcff2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:07.820532 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3821794f-9722-45e9-9d56-0db7f50fcff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c7e2b79f5064252a47372718fe656f4', 'request_id': 'req-3821794f-9722-45e9-9d56-0db7f50fcff2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:07.824773 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:55:07.888177 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3821794f-9722-45e9-9d56-0db7f50fcff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c7e2b79f5064252a47372718fe656f4', 'request_id': 'req-3821794f-9722-45e9-9d56-0db7f50fcff2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:07.899120 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3821794f-9722-45e9-9d56-0db7f50fcff2 admin admin] The token cache is being invalidate because project 8c7e2b79f5064252a47372718fe656f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:55:07.901588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3821794f-9722-45e9-9d56-0db7f50fcff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:07.901588 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3821794f-9722-45e9-9d56-0db7f50fcff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:55:07.902341 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1466/2901] 10.222.0.64 () {58 vars in 1232 bytes} [Mon May 23 15:55:07 2022] DELETE /identity/v3/projects/8c7e2b79f5064252a47372718fe656f4 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:55:16.982337 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=56,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:55:17.974434 dv-cinder-840678-12-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=4,keystone:DELETE=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:55:32.144153 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562c8ba-33e5-4c69-8e33-35a4a0c8e4de admin admin] Authenticating user token {{(pid=80598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:55:32.145723 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-d562c8ba-33e5-4c69-8e33-35a4a0c8e4de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:55:32.212952 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562c8ba-33e5-4c69-8e33-35a4a0c8e4de admin admin] Validating token access rules against request {{(pid=80598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:55:32.269484 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b9e8e-7ecd-4e11-9170-fac4a6fa9530 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fbaeb7583a456d81cf79f6b101fb1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8d81eafac474f37a71ee5cc292d172f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:55:32.270061 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b9e8e-7ecd-4e11-9170-fac4a6fa9530 service placement] REQUEST_METHOD: `GET` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:55:32.270345 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b9e8e-7ecd-4e11-9170-fac4a6fa9530 service placement] SCRIPT_NAME: `/identity` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:55:32.270620 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b9e8e-7ecd-4e11-9170-fac4a6fa9530 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:55:32.272372 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.fernet_utils [None req-cf9b9e8e-7ecd-4e11-9170-fac4a6fa9530 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:55:32.283808 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b9e8e-7ecd-4e11-9170-fac4a6fa9530 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:55:32.285422 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b9e8e-7ecd-4e11-9170-fac4a6fa9530 service placement] RBAC: Authorization granted {{(pid=80598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:55:32.295659 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: [pid: 80598|app: 0|req: 1436/2902] 10.222.0.64 () {60 vars in 1365 bytes} [Mon May 23 15:55:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:55:42.290833 dv-cinder-840678-12-iscsi devstack@keystone.service[80598]: DEBUG dbcounter [-] [80598] Writing DB stats keystone:SELECT=58 {{(pid=80598) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}